[xwiki-devs] [Brainstorming] Migration tool for Nested Pages
Hello everyone. I am working on a tool to help the user migrating from an XWiki instance without the Nested Pages feature to a new version (7.4.x) handling it. This tool would be an extension that the administrator can install and execute *after* the XWiki upgrade have been done. After its execution, users should be able to create children to every existing pages (that was previously terminal, obviously). But the old hierarchy based on the parent/child relationship, and all the preferences and rights, should be preserved. The URL should not be broken neither. I have started a design document with the use-cases I had in mind. I've also suggested some implementations and drew some mock-ups. The results is here: http://design.xwiki.org/xwiki/bin/view/Proposal/UpgradeToNestedPages I've started this document before talking here in order to not come with nothing, but feel free to add other use-cases, ideas, and edge-cases that I haven't listed. We should also identify any blocking point from XWiki Enterprise that we could fix in a 7.4.x version. Do you have any question in mind that we should discuss here? Do you have an opinion on this? You help is warmly welcome, Thanks, -- Guillaume Delhumeau (gdelhumeau@xwiki.com) Research & Development Engineer at XWiki SAS Committer on the XWiki.org project
Hi, The following ideas are about the Parent/Children relation (UC1): For example a copy of design.xwiki.org could be used as a test example for this kind of migration. design.xwiki.org is using a very basic application http://extensions.xwiki.org/xwiki/bin/view/Extension/Proposal+Application that at its core uses the Parent/Child relation in order to list in a livetable the children of a particular proposal. The hierarchy contains pages from 2 spaces: Improvements (from the old application from incubator.myxwiki.org) and Proposal. Sometimes the hierarchy can go to more than 6 levels, examples: http://design.xwiki.org/xwiki/bin/view/Proposal/FlamingoAddMenuLocationItera... or http://design.xwiki.org/xwiki/bin/view/Proposal/XWiki+Icon+Set But sometimes the hierarchy mixes pages from the different spaces, example: http://design.xwiki.org/xwiki/bin/view/Proposal/PanelsImprovements (where Flamingo comes from Improvements space, while the others are from Proposal space). User expectations: 1. I would really like that after the migration the hierarchy to remain the same a.k.a the breadcrumb to be unchanged. The breadcrumb navigation stays at the core of this application. I give more importance to the Page Title than the Page Name, so I would be ok for the new pages to be copied / created using the title and discard the page name. 2. But I sometimes referenciate in the content of some proposals other pages using the page name references. So what will happen to those links? Can we auto-update them inside the content with the new pages (backlinks)? 3. in your UC7: Bookmarks should not be broken after the migration: how can we achieve this? Are the page aliases a solution? Everywhere on jira there are references with links towards the design.xwiki.org proposals. For example, when I moved the app from incubator.myxwiki.org to design.xwiki.org I replaced the content of the page with a Moved type message, example http://incubator.myxwiki.org/xwiki/bin/view/Improvements/XWikiOrg . This could be a solution if we don't provide page aliases or redirects, although might not be very efficient. 4. In order to achieve the hierarchy this means that terminal pages need to be transformed into spaces? 5. In my proposals I use the {{gallery}} macro that uses references like "image:loginDesktopSnapshotAnnotations.png" or "image:Skin4xMenus@menusMenuAnnotations.png", etc. How will the images be affected? 6. Those long URL will look kind of ugly - so again page aliases would be great. But not sure how we will manage them. Do we allow multiple aliases to be created and used, or just 1? 7. What is the complexity of the changes I need to add to the Proposal App in order to continue working as before? Do we provide such documentation for the users? 8. The livetable was using this in order to list the children #set ($discard = $options.put('extraParams', "&parent=${doc.fullName}")) and #set ($importPages = $xwiki.queryManager.xwql("where doc.parent='$doc'").execute()) 9. If the user might start and finish the migration but then he will discover that everything looks bad, what can he do? A best practice would be for him to make a backup before doing the migration, but can't we make one automatically? 10. After the migration what will happen if an user goes to the 'Recycle Bin' and restores a page from "Deleted Documents" or "Deleted Attachments"? 11. Currently there are 1246 pages on the Design wiki. Improvements space contains 317 pages, while Proposal space contains 256 pages. Apparently there are other spaces that contain user generated content like Design, Tech, Standard, etc spaces. It would be great if the migrator could list user created pages when the migrator ask what pages needs to be migrated (UC3). But that list should eliminate default pages and applications pages (technical). Also maybe the user wants to migrate the hierarchy just for some pages (just the spaces Proposal and Improvements). I know maybe the design.xwiki.org is not the best or the most complex example, but as an user I am quite concerned with what will happen to the hierarchy we've created over the years and I'm sure every user thinks the same about their content. Thanks, Caty On Tue, Jan 12, 2016 at 6:37 PM, Guillaume "Louis-Marie" Delhumeau < gdelhumeau@xwiki.com> wrote:
Hello everyone.
I am working on a tool to help the user migrating from an XWiki instance without the Nested Pages feature to a new version (7.4.x) handling it.
This tool would be an extension that the administrator can install and execute *after* the XWiki upgrade have been done. After its execution, users should be able to create children to every existing pages (that was previously terminal, obviously). But the old hierarchy based on the parent/child relationship, and all the preferences and rights, should be preserved. The URL should not be broken neither.
I have started a design document with the use-cases I had in mind. I've also suggested some implementations and drew some mock-ups. The results is here: http://design.xwiki.org/xwiki/bin/view/Proposal/UpgradeToNestedPages
I've started this document before talking here in order to not come with nothing, but feel free to add other use-cases, ideas, and edge-cases that I haven't listed. We should also identify any blocking point from XWiki Enterprise that we could fix in a 7.4.x version.
Do you have any question in mind that we should discuss here? Do you have an opinion on this?
You help is warmly welcome,
Thanks, -- Guillaume Delhumeau (gdelhumeau@xwiki.com) Research & Development Engineer at XWiki SAS Committer on the XWiki.org project _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
12. When I moved the pages from incubator to design.xwiki.org I mostly used scripts. But I had some problems for pages that contain lots of attachments, like http://design.xwiki.org/xwiki/bin/view/Improvements/Skin4x#Attachments Those pages I needed to import with Large import or something, and some even recreated manually. To be honest I don't quite remember, but just be careful that the migrator needs to consider cases when pages cannot be renamed / deleted / etc. Thanks, Caty On Tue, Jan 12, 2016 at 7:26 PM, Ecaterina Moraru (Valica) < valicac@gmail.com> wrote:
Hi,
The following ideas are about the Parent/Children relation (UC1):
For example a copy of design.xwiki.org could be used as a test example for this kind of migration. design.xwiki.org is using a very basic application http://extensions.xwiki.org/xwiki/bin/view/Extension/Proposal+Application that at its core uses the Parent/Child relation in order to list in a livetable the children of a particular proposal.
The hierarchy contains pages from 2 spaces: Improvements (from the old application from incubator.myxwiki.org) and Proposal.
Sometimes the hierarchy can go to more than 6 levels, examples: http://design.xwiki.org/xwiki/bin/view/Proposal/FlamingoAddMenuLocationItera... or http://design.xwiki.org/xwiki/bin/view/Proposal/XWiki+Icon+Set
But sometimes the hierarchy mixes pages from the different spaces, example: http://design.xwiki.org/xwiki/bin/view/Proposal/PanelsImprovements (where Flamingo comes from Improvements space, while the others are from Proposal space).
User expectations: 1. I would really like that after the migration the hierarchy to remain the same a.k.a the breadcrumb to be unchanged. The breadcrumb navigation stays at the core of this application. I give more importance to the Page Title than the Page Name, so I would be ok for the new pages to be copied / created using the title and discard the page name. 2. But I sometimes referenciate in the content of some proposals other pages using the page name references. So what will happen to those links? Can we auto-update them inside the content with the new pages (backlinks)? 3. in your UC7: Bookmarks should not be broken after the migration: how can we achieve this? Are the page aliases a solution? Everywhere on jira there are references with links towards the design.xwiki.org proposals. For example, when I moved the app from incubator.myxwiki.org to design.xwiki.org I replaced the content of the page with a Moved type message, example http://incubator.myxwiki.org/xwiki/bin/view/Improvements/XWikiOrg . This could be a solution if we don't provide page aliases or redirects, although might not be very efficient. 4. In order to achieve the hierarchy this means that terminal pages need to be transformed into spaces? 5. In my proposals I use the {{gallery}} macro that uses references like "image:loginDesktopSnapshotAnnotations.png" or "image:Skin4xMenus@menusMenuAnnotations.png", etc. How will the images be affected? 6. Those long URL will look kind of ugly - so again page aliases would be great. But not sure how we will manage them. Do we allow multiple aliases to be created and used, or just 1? 7. What is the complexity of the changes I need to add to the Proposal App in order to continue working as before? Do we provide such documentation for the users? 8. The livetable was using this in order to list the children #set ($discard = $options.put('extraParams', "& parent=${doc.fullName}")) and #set ($importPages = $xwiki.queryManager.xwql("where doc.parent='$doc'").execute()) 9. If the user might start and finish the migration but then he will discover that everything looks bad, what can he do? A best practice would be for him to make a backup before doing the migration, but can't we make one automatically? 10. After the migration what will happen if an user goes to the 'Recycle Bin' and restores a page from "Deleted Documents" or "Deleted Attachments"? 11. Currently there are 1246 pages on the Design wiki. Improvements space contains 317 pages, while Proposal space contains 256 pages. Apparently there are other spaces that contain user generated content like Design, Tech, Standard, etc spaces. It would be great if the migrator could list user created pages when the migrator ask what pages needs to be migrated (UC3). But that list should eliminate default pages and applications pages (technical). Also maybe the user wants to migrate the hierarchy just for some pages (just the spaces Proposal and Improvements).
I know maybe the design.xwiki.org is not the best or the most complex example, but as an user I am quite concerned with what will happen to the hierarchy we've created over the years and I'm sure every user thinks the same about their content.
Thanks, Caty
On Tue, Jan 12, 2016 at 6:37 PM, Guillaume "Louis-Marie" Delhumeau < gdelhumeau@xwiki.com> wrote:
Hello everyone.
I am working on a tool to help the user migrating from an XWiki instance without the Nested Pages feature to a new version (7.4.x) handling it.
This tool would be an extension that the administrator can install and execute *after* the XWiki upgrade have been done. After its execution, users should be able to create children to every existing pages (that was previously terminal, obviously). But the old hierarchy based on the parent/child relationship, and all the preferences and rights, should be preserved. The URL should not be broken neither.
I have started a design document with the use-cases I had in mind. I've also suggested some implementations and drew some mock-ups. The results is here: http://design.xwiki.org/xwiki/bin/view/Proposal/UpgradeToNestedPages
I've started this document before talking here in order to not come with nothing, but feel free to add other use-cases, ideas, and edge-cases that I haven't listed. We should also identify any blocking point from XWiki Enterprise that we could fix in a 7.4.x version.
Do you have any question in mind that we should discuss here? Do you have an opinion on this?
You help is warmly welcome,
Thanks, -- Guillaume Delhumeau (gdelhumeau@xwiki.com) Research & Development Engineer at XWiki SAS Committer on the XWiki.org project _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
Hi Caty and thanks for your detailed answer. design.xwiki.org seems to be a perfect sample to test and develop this migration tool. Now, let's see point by point. 1. The idea is to have breadcrumbs perfectly identical (ISO) after the migration. However, I was just considering moving the pages from a location to an other, not completely renaming them. I think renaming using the title (which could be velocity code in some places) is out of the scope and could be covered by... an other refactoring tool! 2. Back-links (in the content) will be updated like it usually happens when we rename a page from the UI. We should make sure the tool use the existing mechanism. However, I'm quite sure it won't affect objects properties. 3. Bookmarks won't be broken if we add a redirection to the new page at the old location. Fortunately, Marius have added this feature in 7.4: http://jira.xwiki.org/browse/XWIKI-3622 4. Yes, converting terminal pages to nested pages in the core principle of this tool. It gives the ability to create children to any existing page. Of course, this change should not be done on technical documents, and even in data generated by applications. Otherwise, this application could be broken. 5. I need to see if image references are affected by the backlinks update feature we already have. Same for macro parameters. 6. URL will be long, yes. I don't think URL aliases are planned for 8.0 and it's sure that we won't have them in 7.4.x. 8. Your queries should continue to work, since I propose to update the "parent" field when the parent is renamed. Nested pages will still have the same parent. However, it would be safer to use a query looking at the nested children similar to the one of the children viewer. 9. IMO, we could only propose to generate a full back-up XAR (with history, author preserved, etc...) but it's not the scope of this tool to manage backups. 10. I think it's ok to not handle documents from the recycle bin. Moreover, it will be still possible to run again the tool after some pages have been restored so they eventually reach their ideal location. 11. Adding a white list of spaces to convert could be a great option. For technical documents, for need to create filters. For example: do not convert hidden pages, or pages having some objects of some classes. 12. Moving pages holding a lot of attachments is a problem. See: http://jira.xwiki.org/browse/XWIKI-8910 This tool won't fix bugs that we have for ages. It means we should report any failures during the process in a list that we show to the user. I'm afraid the perfect tool cannot be made and users will still have manual things to do - but at least this tool will make them saving time. Is that OK for you? Any other opinion? Thanks, Guillaume 2016-01-12 18:30 GMT+01:00 Ecaterina Moraru (Valica) <valicac@gmail.com>:
12. When I moved the pages from incubator to design.xwiki.org I mostly used scripts. But I had some problems for pages that contain lots of attachments, like http://design.xwiki.org/xwiki/bin/view/Improvements/Skin4x#Attachments Those pages I needed to import with Large import or something, and some even recreated manually. To be honest I don't quite remember, but just be careful that the migrator needs to consider cases when pages cannot be renamed / deleted / etc.
Thanks, Caty
On Tue, Jan 12, 2016 at 7:26 PM, Ecaterina Moraru (Valica) < valicac@gmail.com> wrote:
Hi,
The following ideas are about the Parent/Children relation (UC1):
For example a copy of design.xwiki.org could be used as a test example for this kind of migration. design.xwiki.org is using a very basic application
http://extensions.xwiki.org/xwiki/bin/view/Extension/Proposal+Application
that at its core uses the Parent/Child relation in order to list in a livetable the children of a particular proposal.
The hierarchy contains pages from 2 spaces: Improvements (from the old application from incubator.myxwiki.org) and Proposal.
Sometimes the hierarchy can go to more than 6 levels, examples:
http://design.xwiki.org/xwiki/bin/view/Proposal/FlamingoAddMenuLocationItera...
or http://design.xwiki.org/xwiki/bin/view/Proposal/XWiki+Icon+Set
But sometimes the hierarchy mixes pages from the different spaces, example: http://design.xwiki.org/xwiki/bin/view/Proposal/PanelsImprovements (where Flamingo comes from Improvements space, while the others are from Proposal space).
User expectations: 1. I would really like that after the migration the hierarchy to remain the same a.k.a the breadcrumb to be unchanged. The breadcrumb navigation stays at the core of this application. I give more importance to the Page Title than the Page Name, so I would be ok for the new pages to be copied / created using the title and discard the page name. 2. But I sometimes referenciate in the content of some proposals other pages using the page name references. So what will happen to those links? Can we auto-update them inside the content with the new pages (backlinks)? 3. in your UC7: Bookmarks should not be broken after the migration: how can we achieve this? Are the page aliases a solution? Everywhere on jira there are references with links towards the design.xwiki.org proposals. For example, when I moved the app from incubator.myxwiki.org to design.xwiki.org I replaced the content of the page with a Moved type message, example http://incubator.myxwiki.org/xwiki/bin/view/Improvements/XWikiOrg . This could be a solution if we don't provide page aliases or redirects, although might not be very efficient. 4. In order to achieve the hierarchy this means that terminal pages need to be transformed into spaces? 5. In my proposals I use the {{gallery}} macro that uses references like "image:loginDesktopSnapshotAnnotations.png" or "image:Skin4xMenus@menusMenuAnnotations.png", etc. How will the images be affected? 6. Those long URL will look kind of ugly - so again page aliases would be great. But not sure how we will manage them. Do we allow multiple aliases to be created and used, or just 1? 7. What is the complexity of the changes I need to add to the Proposal App in order to continue working as before? Do we provide such documentation for the users? 8. The livetable was using this in order to list the children #set ($discard = $options.put('extraParams', "& parent=${doc.fullName}")) and #set ($importPages = $xwiki.queryManager.xwql("where doc.parent='$doc'").execute()) 9. If the user might start and finish the migration but then he will discover that everything looks bad, what can he do? A best practice would be for him to make a backup before doing the migration, but can't we make one automatically? 10. After the migration what will happen if an user goes to the 'Recycle Bin' and restores a page from "Deleted Documents" or "Deleted Attachments"? 11. Currently there are 1246 pages on the Design wiki. Improvements space contains 317 pages, while Proposal space contains 256 pages. Apparently there are other spaces that contain user generated content like Design, Tech, Standard, etc spaces. It would be great if the migrator could list user created pages when the migrator ask what pages needs to be migrated (UC3). But that list should eliminate default pages and applications pages (technical). Also maybe the user wants to migrate the hierarchy just for some pages (just the spaces Proposal and Improvements).
I know maybe the design.xwiki.org is not the best or the most complex example, but as an user I am quite concerned with what will happen to the hierarchy we've created over the years and I'm sure every user thinks the same about their content.
Thanks, Caty
On Tue, Jan 12, 2016 at 6:37 PM, Guillaume "Louis-Marie" Delhumeau < gdelhumeau@xwiki.com> wrote:
Hello everyone.
I am working on a tool to help the user migrating from an XWiki instance without the Nested Pages feature to a new version (7.4.x) handling it.
This tool would be an extension that the administrator can install and execute *after* the XWiki upgrade have been done. After its execution, users should be able to create children to every existing pages (that was previously terminal, obviously). But the old hierarchy based on the parent/child relationship, and all the preferences and rights, should be preserved. The URL should not be broken neither.
I have started a design document with the use-cases I had in mind. I've also suggested some implementations and drew some mock-ups. The results is here: http://design.xwiki.org/xwiki/bin/view/Proposal/UpgradeToNestedPages
I've started this document before talking here in order to not come with nothing, but feel free to add other use-cases, ideas, and edge-cases that I haven't listed. We should also identify any blocking point from XWiki Enterprise that we could fix in a 7.4.x version.
Do you have any question in mind that we should discuss here? Do you have an opinion on this?
You help is warmly welcome,
Thanks, -- Guillaume Delhumeau (gdelhumeau@xwiki.com) Research & Development Engineer at XWiki SAS Committer on the XWiki.org project _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Guillaume Delhumeau (gdelhumeau@xwiki.com) Research & Development Engineer at XWiki SAS Committer on the XWiki.org project
Hi Guillaume, Thanks for your answer. 9. Proposing to create a backup depends on how the user will start the migration tool. If the migration is automatic, than we should create an automated backup. If it's on demand, then we need to suggest to the user to create one before. 12. The issue is for the migration process to not be stuck when encountering problematic pages and at the end to provide a list of failed pages in order for the Administrator to investigate and fix them. Thank you Guillaume, Caty On Wed, Jan 13, 2016 at 1:05 PM, Guillaume "Louis-Marie" Delhumeau < gdelhumeau@xwiki.com> wrote:
Hi Caty and thanks for your detailed answer. design.xwiki.org seems to be a perfect sample to test and develop this migration tool.
Now, let's see point by point.
1. The idea is to have breadcrumbs perfectly identical (ISO) after the migration. However, I was just considering moving the pages from a location to an other, not completely renaming them. I think renaming using the title (which could be velocity code in some places) is out of the scope and could be covered by... an other refactoring tool!
2. Back-links (in the content) will be updated like it usually happens when we rename a page from the UI. We should make sure the tool use the existing mechanism. However, I'm quite sure it won't affect objects properties.
3. Bookmarks won't be broken if we add a redirection to the new page at the old location. Fortunately, Marius have added this feature in 7.4: http://jira.xwiki.org/browse/XWIKI-3622
4. Yes, converting terminal pages to nested pages in the core principle of this tool. It gives the ability to create children to any existing page. Of course, this change should not be done on technical documents, and even in data generated by applications. Otherwise, this application could be broken.
5. I need to see if image references are affected by the backlinks update feature we already have. Same for macro parameters.
6. URL will be long, yes. I don't think URL aliases are planned for 8.0 and it's sure that we won't have them in 7.4.x.
8. Your queries should continue to work, since I propose to update the "parent" field when the parent is renamed. Nested pages will still have the same parent. However, it would be safer to use a query looking at the nested children similar to the one of the children viewer.
9. IMO, we could only propose to generate a full back-up XAR (with history, author preserved, etc...) but it's not the scope of this tool to manage backups.
10. I think it's ok to not handle documents from the recycle bin. Moreover, it will be still possible to run again the tool after some pages have been restored so they eventually reach their ideal location.
11. Adding a white list of spaces to convert could be a great option. For technical documents, for need to create filters. For example: do not convert hidden pages, or pages having some objects of some classes.
12. Moving pages holding a lot of attachments is a problem. See: http://jira.xwiki.org/browse/XWIKI-8910
This tool won't fix bugs that we have for ages. It means we should report any failures during the process in a list that we show to the user. I'm afraid the perfect tool cannot be made and users will still have manual things to do - but at least this tool will make them saving time.
Is that OK for you?
Any other opinion?
Thanks, Guillaume
2016-01-12 18:30 GMT+01:00 Ecaterina Moraru (Valica) <valicac@gmail.com>:
12. When I moved the pages from incubator to design.xwiki.org I mostly used scripts. But I had some problems for pages that contain lots of attachments, like http://design.xwiki.org/xwiki/bin/view/Improvements/Skin4x#Attachments Those pages I needed to import with Large import or something, and some even recreated manually. To be honest I don't quite remember, but just be careful that the migrator needs to consider cases when pages cannot be renamed / deleted / etc.
Thanks, Caty
On Tue, Jan 12, 2016 at 7:26 PM, Ecaterina Moraru (Valica) < valicac@gmail.com> wrote:
Hi,
The following ideas are about the Parent/Children relation (UC1):
For example a copy of design.xwiki.org could be used as a test example for this kind of migration. design.xwiki.org is using a very basic application
http://extensions.xwiki.org/xwiki/bin/view/Extension/Proposal+Application
that at its core uses the Parent/Child relation in order to list in a livetable the children of a particular proposal.
The hierarchy contains pages from 2 spaces: Improvements (from the old application from incubator.myxwiki.org) and Proposal.
Sometimes the hierarchy can go to more than 6 levels, examples:
http://design.xwiki.org/xwiki/bin/view/Proposal/FlamingoAddMenuLocationItera...
or http://design.xwiki.org/xwiki/bin/view/Proposal/XWiki+Icon+Set
But sometimes the hierarchy mixes pages from the different spaces, example: http://design.xwiki.org/xwiki/bin/view/Proposal/PanelsImprovements (where Flamingo comes from Improvements space, while the others are from Proposal space).
User expectations: 1. I would really like that after the migration the hierarchy to remain the same a.k.a the breadcrumb to be unchanged. The breadcrumb navigation stays at the core of this application. I give more importance to the Page Title than the Page Name, so I would be ok for the new pages to be copied / created using the title and discard the page name. 2. But I sometimes referenciate in the content of some proposals other pages using the page name references. So what will happen to those links? Can we auto-update them inside the content with the new pages (backlinks)? 3. in your UC7: Bookmarks should not be broken after the migration: how can we achieve this? Are the page aliases a solution? Everywhere on jira there are references with links towards the design.xwiki.org proposals. For example, when I moved the app from incubator.myxwiki.org to design.xwiki.org I replaced the content of the page with a Moved type message, example http://incubator.myxwiki.org/xwiki/bin/view/Improvements/XWikiOrg . This could be a solution if we don't provide page aliases or redirects, although might not be very efficient. 4. In order to achieve the hierarchy this means that terminal pages need to be transformed into spaces? 5. In my proposals I use the {{gallery}} macro that uses references like "image:loginDesktopSnapshotAnnotations.png" or "image:Skin4xMenus@menusMenuAnnotations.png", etc. How will the images be affected? 6. Those long URL will look kind of ugly - so again page aliases would be great. But not sure how we will manage them. Do we allow multiple aliases to be created and used, or just 1? 7. What is the complexity of the changes I need to add to the Proposal App in order to continue working as before? Do we provide such documentation for the users? 8. The livetable was using this in order to list the children #set ($discard = $options.put('extraParams', "& parent=${doc.fullName}")) and #set ($importPages = $xwiki.queryManager.xwql("where doc.parent='$doc'").execute()) 9. If the user might start and finish the migration but then he will discover that everything looks bad, what can he do? A best practice would be for him to make a backup before doing the migration, but can't we make one automatically? 10. After the migration what will happen if an user goes to the 'Recycle Bin' and restores a page from "Deleted Documents" or "Deleted Attachments"? 11. Currently there are 1246 pages on the Design wiki. Improvements space contains 317 pages, while Proposal space contains 256 pages. Apparently there are other spaces that contain user generated content like Design, Tech, Standard, etc spaces. It would be great if the migrator could list user created pages when the migrator ask what pages needs to be migrated (UC3). But that list should eliminate default pages and applications pages (technical). Also maybe the user wants to migrate the hierarchy just for some pages (just the spaces Proposal and Improvements).
I know maybe the design.xwiki.org is not the best or the most complex example, but as an user I am quite concerned with what will happen to the hierarchy we've created over the years and I'm sure every user thinks the same about their content.
Thanks, Caty
On Tue, Jan 12, 2016 at 6:37 PM, Guillaume "Louis-Marie" Delhumeau < gdelhumeau@xwiki.com> wrote:
Hello everyone.
I am working on a tool to help the user migrating from an XWiki instance without the Nested Pages feature to a new version (7.4.x) handling it.
This tool would be an extension that the administrator can install and execute *after* the XWiki upgrade have been done. After its execution, users should be able to create children to every existing pages (that was previously terminal, obviously). But the old hierarchy based on the parent/child relationship, and all the preferences and rights, should be preserved. The URL should not be broken neither.
I have started a design document with the use-cases I had in mind. I've also suggested some implementations and drew some mock-ups. The results is here: http://design.xwiki.org/xwiki/bin/view/Proposal/UpgradeToNestedPages
I've started this document before talking here in order to not come with nothing, but feel free to add other use-cases, ideas, and edge-cases that I haven't listed. We should also identify any blocking point from XWiki Enterprise that we could fix in a 7.4.x version.
Do you have any question in mind that we should discuss here? Do you have an opinion on this?
You help is warmly welcome,
Thanks, -- Guillaume Delhumeau (gdelhumeau@xwiki.com) Research & Development Engineer at XWiki SAS Committer on the XWiki.org project _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Guillaume Delhumeau (gdelhumeau@xwiki.com) Research & Development Engineer at XWiki SAS Committer on the XWiki.org project _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
Hi. I have released a first alpha version of the application: http://extensions.xwiki.org/xwiki/bin/view/Extension/Nested+Pages+Migrator+A... This application allows to compute a plan of actions to do to perform a migration from terminal pages to nested pages. The current version does not propose to apply this plan so it is only available for testing purpose. Current limitations: - The plan is not applied yet. - The preferences and the configured rights are not converted yet. - 2 pages can have the same target location. - There is no real pagination You should test it on a wiki containing a lot of documents. You need to upgrade your wiki to a version handling Nested Pages before using the application. Sometimes, the parent/based relationship between pages is messy. For example, I have run this script before using my tool on a local copy of design.xwiki.org because a lot of pages had Main.WebHome as parent meanwhile it was not justified: {{velocity}} #set ($xwql = "where doc.space in ('Proposal', 'Design', 'Improvements') and doc.parent = 'Main.WebHome'") #foreach($r in $services.query.xwql($xwql).execute()) * $r #set ($d = $xwiki.getDocument($r)) #set ($discard = $d.setParent('Proposal.WebHome')) #set ($discard = $d.save()) #end {{/velocity}} The UI is a single-page application that handles some options that I let you discover. I hope you will have some time to test it and that you will like the tool. Thanks for your feedbacks, Guillaume 2016-01-13 12:54 GMT+01:00 Ecaterina Moraru (Valica) <valicac@gmail.com>:
Hi Guillaume,
Thanks for your answer.
9. Proposing to create a backup depends on how the user will start the migration tool. If the migration is automatic, than we should create an automated backup. If it's on demand, then we need to suggest to the user to create one before.
12. The issue is for the migration process to not be stuck when encountering problematic pages and at the end to provide a list of failed pages in order for the Administrator to investigate and fix them.
Thank you Guillaume, Caty
On Wed, Jan 13, 2016 at 1:05 PM, Guillaume "Louis-Marie" Delhumeau < gdelhumeau@xwiki.com> wrote:
Hi Caty and thanks for your detailed answer. design.xwiki.org seems to be a perfect sample to test and develop this migration tool.
Now, let's see point by point.
1. The idea is to have breadcrumbs perfectly identical (ISO) after the migration. However, I was just considering moving the pages from a location to an other, not completely renaming them. I think renaming using the title (which could be velocity code in some places) is out of the scope and could be covered by... an other refactoring tool!
2. Back-links (in the content) will be updated like it usually happens when we rename a page from the UI. We should make sure the tool use the existing mechanism. However, I'm quite sure it won't affect objects properties.
3. Bookmarks won't be broken if we add a redirection to the new page at the old location. Fortunately, Marius have added this feature in 7.4: http://jira.xwiki.org/browse/XWIKI-3622
4. Yes, converting terminal pages to nested pages in the core principle of this tool. It gives the ability to create children to any existing page. Of course, this change should not be done on technical documents, and even in data generated by applications. Otherwise, this application could be broken.
5. I need to see if image references are affected by the backlinks update feature we already have. Same for macro parameters.
6. URL will be long, yes. I don't think URL aliases are planned for 8.0 and it's sure that we won't have them in 7.4.x.
8. Your queries should continue to work, since I propose to update the "parent" field when the parent is renamed. Nested pages will still have the same parent. However, it would be safer to use a query looking at the nested children similar to the one of the children viewer.
9. IMO, we could only propose to generate a full back-up XAR (with history, author preserved, etc...) but it's not the scope of this tool to manage backups.
10. I think it's ok to not handle documents from the recycle bin. Moreover, it will be still possible to run again the tool after some pages have been restored so they eventually reach their ideal location.
11. Adding a white list of spaces to convert could be a great option. For technical documents, for need to create filters. For example: do not convert hidden pages, or pages having some objects of some classes.
12. Moving pages holding a lot of attachments is a problem. See: http://jira.xwiki.org/browse/XWIKI-8910
This tool won't fix bugs that we have for ages. It means we should report any failures during the process in a list that we show to the user. I'm afraid the perfect tool cannot be made and users will still have manual things to do - but at least this tool will make them saving time.
Is that OK for you?
Any other opinion?
Thanks, Guillaume
2016-01-12 18:30 GMT+01:00 Ecaterina Moraru (Valica) <valicac@gmail.com :
12. When I moved the pages from incubator to design.xwiki.org I mostly used scripts. But I had some problems for pages that contain lots of attachments, like http://design.xwiki.org/xwiki/bin/view/Improvements/Skin4x#Attachments Those pages I needed to import with Large import or something, and some even recreated manually. To be honest I don't quite remember, but just be careful that the migrator needs to consider cases when pages cannot be renamed / deleted / etc.
Thanks, Caty
On Tue, Jan 12, 2016 at 7:26 PM, Ecaterina Moraru (Valica) < valicac@gmail.com> wrote:
Hi,
The following ideas are about the Parent/Children relation (UC1):
For example a copy of design.xwiki.org could be used as a test example for this kind of migration. design.xwiki.org is using a very basic application
http://extensions.xwiki.org/xwiki/bin/view/Extension/Proposal+Application
that at its core uses the Parent/Child relation in order to list in a livetable the children of a particular proposal.
The hierarchy contains pages from 2 spaces: Improvements (from the old application from incubator.myxwiki.org) and Proposal.
Sometimes the hierarchy can go to more than 6 levels, examples:
http://design.xwiki.org/xwiki/bin/view/Proposal/FlamingoAddMenuLocationItera...
or http://design.xwiki.org/xwiki/bin/view/Proposal/XWiki+Icon+Set
But sometimes the hierarchy mixes pages from the different spaces, example: http://design.xwiki.org/xwiki/bin/view/Proposal/PanelsImprovements (where Flamingo comes from Improvements space, while the others are from Proposal space).
User expectations: 1. I would really like that after the migration the hierarchy to remain the same a.k.a the breadcrumb to be unchanged. The breadcrumb navigation stays at the core of this application. I give more importance to the Page Title than the Page Name, so I would be ok for the new pages to be copied / created using the title and discard the page name. 2. But I sometimes referenciate in the content of some proposals other pages using the page name references. So what will happen to those links? Can we auto-update them inside the content with the new pages (backlinks)? 3. in your UC7: Bookmarks should not be broken after the migration: how can we achieve this? Are the page aliases a solution? Everywhere on jira there are references with links towards the design.xwiki.org proposals. For example, when I moved the app from incubator.myxwiki.org to design.xwiki.org I replaced the content of the page with a Moved type message, example http://incubator.myxwiki.org/xwiki/bin/view/Improvements/XWikiOrg . This could be a solution if we don't provide page aliases or redirects, although might not be very efficient. 4. In order to achieve the hierarchy this means that terminal pages need to be transformed into spaces? 5. In my proposals I use the {{gallery}} macro that uses references like "image:loginDesktopSnapshotAnnotations.png" or "image:Skin4xMenus@menusMenuAnnotations.png", etc. How will the images be affected? 6. Those long URL will look kind of ugly - so again page aliases would be great. But not sure how we will manage them. Do we allow multiple aliases to be created and used, or just 1? 7. What is the complexity of the changes I need to add to the Proposal App in order to continue working as before? Do we provide such documentation for the users? 8. The livetable was using this in order to list the children #set ($discard = $options.put('extraParams', "& parent=${doc.fullName}")) and #set ($importPages = $xwiki.queryManager.xwql("where doc.parent='$doc'").execute()) 9. If the user might start and finish the migration but then he will discover that everything looks bad, what can he do? A best practice would be for him to make a backup before doing the migration, but can't we make one automatically? 10. After the migration what will happen if an user goes to the 'Recycle Bin' and restores a page from "Deleted Documents" or "Deleted Attachments"? 11. Currently there are 1246 pages on the Design wiki. Improvements space contains 317 pages, while Proposal space contains 256 pages. Apparently there are other spaces that contain user generated content like Design, Tech, Standard, etc spaces. It would be great if the migrator could list user created pages when the migrator ask what pages needs to be migrated (UC3). But that list should eliminate default pages and applications pages (technical). Also maybe the user wants to migrate the hierarchy just for some pages (just the spaces Proposal and Improvements).
I know maybe the design.xwiki.org is not the best or the most complex example, but as an user I am quite concerned with what will happen to the hierarchy we've created over the years and I'm sure every user thinks the same about their content.
Thanks, Caty
On Tue, Jan 12, 2016 at 6:37 PM, Guillaume "Louis-Marie" Delhumeau < gdelhumeau@xwiki.com> wrote:
Hello everyone.
I am working on a tool to help the user migrating from an XWiki instance without the Nested Pages feature to a new version (7.4.x) handling it.
This tool would be an extension that the administrator can install and execute *after* the XWiki upgrade have been done. After its execution, users should be able to create children to every existing pages (that was previously terminal, obviously). But the old hierarchy based on the parent/child relationship, and all the preferences and rights, should be preserved. The URL should not be broken neither.
I have started a design document with the use-cases I had in mind. I've also suggested some implementations and drew some mock-ups. The results is here:
http://design.xwiki.org/xwiki/bin/view/Proposal/UpgradeToNestedPages
I've started this document before talking here in order to not come
with
nothing, but feel free to add other use-cases, ideas, and edge-cases that I haven't listed. We should also identify any blocking point from XWiki Enterprise that we could fix in a 7.4.x version.
Do you have any question in mind that we should discuss here? Do you have an opinion on this?
You help is warmly welcome,
Thanks, -- Guillaume Delhumeau (gdelhumeau@xwiki.com) Research & Development Engineer at XWiki SAS Committer on the XWiki.org project _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Guillaume Delhumeau (gdelhumeau@xwiki.com) Research & Development Engineer at XWiki SAS Committer on the XWiki.org project _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Guillaume Delhumeau (gdelhumeau@xwiki.com) Research & Development Engineer at XWiki SAS Committer on the XWiki.org project
It looks good from a parent/child perspective, especially for the URL preserving and the redirect. Thanks Guillaume for your work. Now I guess the migrator should also cover/test the rights (unfortunately design.xwiki.org doesn't have anything special regarding Rights to you might need another test case). Thanks, Caty On Tue, Jan 19, 2016 at 7:34 PM, Guillaume "Louis-Marie" Delhumeau < gdelhumeau@xwiki.com> wrote:
Hi.
I have released a first alpha version of the application:
http://extensions.xwiki.org/xwiki/bin/view/Extension/Nested+Pages+Migrator+A...
This application allows to compute a plan of actions to do to perform a migration from terminal pages to nested pages. The current version does not propose to apply this plan so it is only available for testing purpose.
Current limitations:
- The plan is not applied yet. - The preferences and the configured rights are not converted yet. - 2 pages can have the same target location. - There is no real pagination
You should test it on a wiki containing a lot of documents. You need to upgrade your wiki to a version handling Nested Pages before using the application.
Sometimes, the parent/based relationship between pages is messy. For example, I have run this script before using my tool on a local copy of design.xwiki.org because a lot of pages had Main.WebHome as parent meanwhile it was not justified:
{{velocity}} #set ($xwql = "where doc.space in ('Proposal', 'Design', 'Improvements') and doc.parent = 'Main.WebHome'") #foreach($r in $services.query.xwql($xwql).execute()) * $r #set ($d = $xwiki.getDocument($r)) #set ($discard = $d.setParent('Proposal.WebHome')) #set ($discard = $d.save()) #end {{/velocity}}
The UI is a single-page application that handles some options that I let you discover.
I hope you will have some time to test it and that you will like the tool. Thanks for your feedbacks,
Guillaume
2016-01-13 12:54 GMT+01:00 Ecaterina Moraru (Valica) <valicac@gmail.com>:
Hi Guillaume,
Thanks for your answer.
9. Proposing to create a backup depends on how the user will start the migration tool. If the migration is automatic, than we should create an automated backup. If it's on demand, then we need to suggest to the user to create one before.
12. The issue is for the migration process to not be stuck when encountering problematic pages and at the end to provide a list of failed pages in order for the Administrator to investigate and fix them.
Thank you Guillaume, Caty
On Wed, Jan 13, 2016 at 1:05 PM, Guillaume "Louis-Marie" Delhumeau < gdelhumeau@xwiki.com> wrote:
Hi Caty and thanks for your detailed answer. design.xwiki.org seems to be a perfect sample to test and develop this migration tool.
Now, let's see point by point.
1. The idea is to have breadcrumbs perfectly identical (ISO) after the migration. However, I was just considering moving the pages from a location to an other, not completely renaming them. I think renaming using the title (which could be velocity code in some places) is out of the scope and could be covered by... an other refactoring tool!
2. Back-links (in the content) will be updated like it usually happens when we rename a page from the UI. We should make sure the tool use the existing mechanism. However, I'm quite sure it won't affect objects properties.
3. Bookmarks won't be broken if we add a redirection to the new page at the old location. Fortunately, Marius have added this feature in 7.4: http://jira.xwiki.org/browse/XWIKI-3622
4. Yes, converting terminal pages to nested pages in the core principle of this tool. It gives the ability to create children to any existing page. Of course, this change should not be done on technical documents, and even in data generated by applications. Otherwise, this application could be broken.
5. I need to see if image references are affected by the backlinks update feature we already have. Same for macro parameters.
6. URL will be long, yes. I don't think URL aliases are planned for 8.0 and it's sure that we won't have them in 7.4.x.
8. Your queries should continue to work, since I propose to update the "parent" field when the parent is renamed. Nested pages will still have the same parent. However, it would be safer to use a query looking at the nested children similar to the one of the children viewer.
9. IMO, we could only propose to generate a full back-up XAR (with history, author preserved, etc...) but it's not the scope of this tool to manage backups.
10. I think it's ok to not handle documents from the recycle bin. Moreover, it will be still possible to run again the tool after some pages have been restored so they eventually reach their ideal location.
11. Adding a white list of spaces to convert could be a great option. For technical documents, for need to create filters. For example: do not convert hidden pages, or pages having some objects of some classes.
12. Moving pages holding a lot of attachments is a problem. See: http://jira.xwiki.org/browse/XWIKI-8910
This tool won't fix bugs that we have for ages. It means we should report any failures during the process in a list that we show to the user. I'm afraid the perfect tool cannot be made and users will still have manual things to do - but at least this tool will make them saving time.
Is that OK for you?
Any other opinion?
Thanks, Guillaume
2016-01-12 18:30 GMT+01:00 Ecaterina Moraru (Valica) < valicac@gmail.com :
12. When I moved the pages from incubator to design.xwiki.org I mostly used scripts. But I had some problems for pages that contain lots of attachments, like
http://design.xwiki.org/xwiki/bin/view/Improvements/Skin4x#Attachments
Those pages I needed to import with Large import or something, and some even recreated manually. To be honest I don't quite remember, but just be careful that the migrator needs to consider cases when pages cannot be renamed / deleted / etc.
Thanks, Caty
On Tue, Jan 12, 2016 at 7:26 PM, Ecaterina Moraru (Valica) < valicac@gmail.com> wrote:
Hi,
The following ideas are about the Parent/Children relation (UC1):
For example a copy of design.xwiki.org could be used as a test example for this kind of migration. design.xwiki.org is using a very basic application
http://extensions.xwiki.org/xwiki/bin/view/Extension/Proposal+Application
that at its core uses the Parent/Child relation in order to list in a livetable the children of a particular proposal.
The hierarchy contains pages from 2 spaces: Improvements (from the old application from incubator.myxwiki.org) and Proposal.
Sometimes the hierarchy can go to more than 6 levels, examples:
http://design.xwiki.org/xwiki/bin/view/Proposal/FlamingoAddMenuLocationItera...
or http://design.xwiki.org/xwiki/bin/view/Proposal/XWiki+Icon+Set
But sometimes the hierarchy mixes pages from the different spaces, example: http://design.xwiki.org/xwiki/bin/view/Proposal/PanelsImprovements (where Flamingo comes from Improvements space, while the others are from Proposal space).
User expectations: 1. I would really like that after the migration the hierarchy to remain the same a.k.a the breadcrumb to be unchanged. The breadcrumb navigation stays at the core of this application. I give more importance to the Page Title than the Page Name, so I would be ok for the new pages to be copied / created using the title and discard the page name. 2. But I sometimes referenciate in the content of some proposals other pages using the page name references. So what will happen to those links? Can we auto-update them inside the content with the new pages (backlinks)? 3. in your UC7: Bookmarks should not be broken after the migration: how can we achieve this? Are the page aliases a solution? Everywhere on jira there are references with links towards the design.xwiki.org proposals. For example, when I moved the app from incubator.myxwiki.org to design.xwiki.org I replaced the content of the page with a Moved type message, example http://incubator.myxwiki.org/xwiki/bin/view/Improvements/XWikiOrg . This could be a solution if we don't provide page aliases or redirects, although might not be very efficient. 4. In order to achieve the hierarchy this means that terminal pages need to be transformed into spaces? 5. In my proposals I use the {{gallery}} macro that uses references like "image:loginDesktopSnapshotAnnotations.png" or "image:Skin4xMenus@menusMenuAnnotations.png", etc. How will the images be affected? 6. Those long URL will look kind of ugly - so again page aliases would be great. But not sure how we will manage them. Do we allow multiple aliases to be created and used, or just 1? 7. What is the complexity of the changes I need to add to the Proposal App in order to continue working as before? Do we provide such documentation for the users? 8. The livetable was using this in order to list the children #set ($discard = $options.put('extraParams', "& parent=${doc.fullName}")) and #set ($importPages = $xwiki.queryManager.xwql("where doc.parent='$doc'").execute()) 9. If the user might start and finish the migration but then he will discover that everything looks bad, what can he do? A best practice would be for him to make a backup before doing the migration, but can't we make one automatically? 10. After the migration what will happen if an user goes to the 'Recycle Bin' and restores a page from "Deleted Documents" or "Deleted Attachments"? 11. Currently there are 1246 pages on the Design wiki. Improvements space contains 317 pages, while Proposal space contains 256 pages. Apparently there are other spaces that contain user generated content like Design, Tech, Standard, etc spaces. It would be great if the migrator could list user created pages when the migrator ask what pages needs to be migrated (UC3). But that list should eliminate default pages and applications pages (technical). Also maybe the user wants to migrate the hierarchy just for some pages (just the spaces Proposal and Improvements).
I know maybe the design.xwiki.org is not the best or the most complex example, but as an user I am quite concerned with what will happen to the hierarchy we've created over the years and I'm sure every user thinks the same about their content.
Thanks, Caty
On Tue, Jan 12, 2016 at 6:37 PM, Guillaume "Louis-Marie" Delhumeau < gdelhumeau@xwiki.com> wrote:
Hello everyone.
I am working on a tool to help the user migrating from an XWiki instance without the Nested Pages feature to a new version (7.4.x) handling it.
This tool would be an extension that the administrator can install and execute *after* the XWiki upgrade have been done. After its execution, users should be able to create children to every existing pages (that was previously terminal, obviously). But the old hierarchy based on the parent/child relationship, and all the preferences and rights, should be preserved. The URL should not be broken neither.
I have started a design document with the use-cases I had in mind. I've also suggested some implementations and drew some mock-ups. The results is here:
http://design.xwiki.org/xwiki/bin/view/Proposal/UpgradeToNestedPages
I've started this document before talking here in order to not
come with
nothing, but feel free to add other use-cases, ideas, and edge-cases that I haven't listed. We should also identify any blocking point from XWiki Enterprise that we could fix in a 7.4.x version.
Do you have any question in mind that we should discuss here? Do you have an opinion on this?
You help is warmly welcome,
Thanks, -- Guillaume Delhumeau (gdelhumeau@xwiki.com) Research & Development Engineer at XWiki SAS Committer on the XWiki.org project _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Guillaume Delhumeau (gdelhumeau@xwiki.com) Research & Development Engineer at XWiki SAS Committer on the XWiki.org project _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Guillaume Delhumeau (gdelhumeau@xwiki.com) Research & Development Engineer at XWiki SAS Committer on the XWiki.org project _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
Cool Guillaume, maybe you should send that mail to the user list since we could get feedback from users trying to migrate to 7.4? Thanks -Vincent On 19 Jan 2016 at 18:34:39, Guillaume Louis-Marie Delhumeau (gdelhumeau@xwiki.com) wrote: Hi. I have released a first alpha version of the application: http://extensions.xwiki.org/xwiki/bin/view/Extension/Nested+Pages+Migrator+A... This application allows to compute a plan of actions to do to perform a migration from terminal pages to nested pages. The current version does not propose to apply this plan so it is only available for testing purpose. Current limitations: - The plan is not applied yet. - The preferences and the configured rights are not converted yet. - 2 pages can have the same target location. - There is no real pagination You should test it on a wiki containing a lot of documents. You need to upgrade your wiki to a version handling Nested Pages before using the application. Sometimes, the parent/based relationship between pages is messy. For example, I have run this script before using my tool on a local copy of design.xwiki.org because a lot of pages had Main.WebHome as parent meanwhile it was not justified: {{velocity}} #set ($xwql = "where doc.space in ('Proposal', 'Design', 'Improvements') and doc.parent = 'Main.WebHome'") #foreach($r in $services.query.xwql($xwql).execute()) * $r #set ($d = $xwiki.getDocument($r)) #set ($discard = $d.setParent('Proposal.WebHome')) #set ($discard = $d.save()) #end {{/velocity}} The UI is a single-page application that handles some options that I let you discover. I hope you will have some time to test it and that you will like the tool. Thanks for your feedbacks, Guillaume 2016-01-13 12:54 GMT+01:00 Ecaterina Moraru (Valica) <valicac@gmail.com>:
Hi Guillaume,
Thanks for your answer.
9. Proposing to create a backup depends on how the user will start the migration tool. If the migration is automatic, than we should create an automated backup. If it's on demand, then we need to suggest to the user to create one before.
12. The issue is for the migration process to not be stuck when encountering problematic pages and at the end to provide a list of failed pages in order for the Administrator to investigate and fix them.
Thank you Guillaume, Caty
On Wed, Jan 13, 2016 at 1:05 PM, Guillaume "Louis-Marie" Delhumeau < gdelhumeau@xwiki.com> wrote:
Hi Caty and thanks for your detailed answer. design.xwiki.org seems to be a perfect sample to test and develop this migration tool.
Now, let's see point by point.
1. The idea is to have breadcrumbs perfectly identical (ISO) after the migration. However, I was just considering moving the pages from a location to an other, not completely renaming them. I think renaming using the title (which could be velocity code in some places) is out of the scope and could be covered by... an other refactoring tool!
2. Back-links (in the content) will be updated like it usually happens when we rename a page from the UI. We should make sure the tool use the existing mechanism. However, I'm quite sure it won't affect objects properties.
3. Bookmarks won't be broken if we add a redirection to the new page at the old location. Fortunately, Marius have added this feature in 7.4: http://jira.xwiki.org/browse/XWIKI-3622
4. Yes, converting terminal pages to nested pages in the core principle of this tool. It gives the ability to create children to any existing page. Of course, this change should not be done on technical documents, and even in data generated by applications. Otherwise, this application could be broken.
5. I need to see if image references are affected by the backlinks update feature we already have. Same for macro parameters.
6. URL will be long, yes. I don't think URL aliases are planned for 8.0 and it's sure that we won't have them in 7.4.x.
8. Your queries should continue to work, since I propose to update the "parent" field when the parent is renamed. Nested pages will still have the same parent. However, it would be safer to use a query looking at the nested children similar to the one of the children viewer.
9. IMO, we could only propose to generate a full back-up XAR (with history, author preserved, etc...) but it's not the scope of this tool to manage backups.
10. I think it's ok to not handle documents from the recycle bin. Moreover, it will be still possible to run again the tool after some pages have been restored so they eventually reach their ideal location.
11. Adding a white list of spaces to convert could be a great option. For technical documents, for need to create filters. For example: do not convert hidden pages, or pages having some objects of some classes.
12. Moving pages holding a lot of attachments is a problem. See: http://jira.xwiki.org/browse/XWIKI-8910
This tool won't fix bugs that we have for ages. It means we should report any failures during the process in a list that we show to the user. I'm afraid the perfect tool cannot be made and users will still have manual things to do - but at least this tool will make them saving time.
Is that OK for you?
Any other opinion?
Thanks, Guillaume
2016-01-12 18:30 GMT+01:00 Ecaterina Moraru (Valica) <valicac@gmail.com :
12. When I moved the pages from incubator to design.xwiki.org I mostly used scripts. But I had some problems for pages that contain lots of attachments, like http://design.xwiki.org/xwiki/bin/view/Improvements/Skin4x#Attachments Those pages I needed to import with Large import or something, and some even recreated manually. To be honest I don't quite remember, but just be careful that the migrator needs to consider cases when pages cannot be renamed / deleted / etc.
Thanks, Caty
On Tue, Jan 12, 2016 at 7:26 PM, Ecaterina Moraru (Valica) < valicac@gmail.com> wrote:
Hi,
The following ideas are about the Parent/Children relation (UC1):
For example a copy of design.xwiki.org could be used as a test example for this kind of migration. design.xwiki.org is using a very basic application
http://extensions.xwiki.org/xwiki/bin/view/Extension/Proposal+Application
that at its core uses the Parent/Child relation in order to list in a livetable the children of a particular proposal.
The hierarchy contains pages from 2 spaces: Improvements (from the old application from incubator.myxwiki.org) and Proposal.
Sometimes the hierarchy can go to more than 6 levels, examples:
http://design.xwiki.org/xwiki/bin/view/Proposal/FlamingoAddMenuLocationItera...
or http://design.xwiki.org/xwiki/bin/view/Proposal/XWiki+Icon+Set
But sometimes the hierarchy mixes pages from the different spaces, example: http://design.xwiki.org/xwiki/bin/view/Proposal/PanelsImprovements (where Flamingo comes from Improvements space, while the others are from Proposal space).
User expectations: 1. I would really like that after the migration the hierarchy to remain the same a.k.a the breadcrumb to be unchanged. The breadcrumb navigation stays at the core of this application. I give more importance to the Page Title than the Page Name, so I would be ok for the new pages to be copied / created using the title and discard the page name. 2. But I sometimes referenciate in the content of some proposals other pages using the page name references. So what will happen to those links? Can we auto-update them inside the content with the new pages (backlinks)? 3. in your UC7: Bookmarks should not be broken after the migration: how can we achieve this? Are the page aliases a solution? Everywhere on jira there are references with links towards the design.xwiki.org proposals. For example, when I moved the app from incubator.myxwiki.org to design.xwiki.org I replaced the content of the page with a Moved type message, example http://incubator.myxwiki.org/xwiki/bin/view/Improvements/XWikiOrg . This could be a solution if we don't provide page aliases or redirects, although might not be very efficient. 4. In order to achieve the hierarchy this means that terminal pages need to be transformed into spaces? 5. In my proposals I use the {{gallery}} macro that uses references like "image:loginDesktopSnapshotAnnotations.png" or "image:Skin4xMenus@menusMenuAnnotations.png", etc. How will the images be affected? 6. Those long URL will look kind of ugly - so again page aliases would be great. But not sure how we will manage them. Do we allow multiple aliases to be created and used, or just 1? 7. What is the complexity of the changes I need to add to the Proposal App in order to continue working as before? Do we provide such documentation for the users? 8. The livetable was using this in order to list the children #set ($discard = $options.put('extraParams', "& parent=${doc.fullName}")) and #set ($importPages = $xwiki.queryManager.xwql("where doc.parent='$doc'").execute()) 9. If the user might start and finish the migration but then he will discover that everything looks bad, what can he do? A best practice would be for him to make a backup before doing the migration, but can't we make one automatically? 10. After the migration what will happen if an user goes to the 'Recycle Bin' and restores a page from "Deleted Documents" or "Deleted Attachments"? 11. Currently there are 1246 pages on the Design wiki. Improvements space contains 317 pages, while Proposal space contains 256 pages. Apparently there are other spaces that contain user generated content like Design, Tech, Standard, etc spaces. It would be great if the migrator could list user created pages when the migrator ask what pages needs to be migrated (UC3). But that list should eliminate default pages and applications pages (technical). Also maybe the user wants to migrate the hierarchy just for some pages (just the spaces Proposal and Improvements).
I know maybe the design.xwiki.org is not the best or the most complex example, but as an user I am quite concerned with what will happen to the hierarchy we've created over the years and I'm sure every user thinks the same about their content.
Thanks, Caty
On Tue, Jan 12, 2016 at 6:37 PM, Guillaume "Louis-Marie" Delhumeau < gdelhumeau@xwiki.com> wrote:
Hello everyone.
I am working on a tool to help the user migrating from an XWiki instance without the Nested Pages feature to a new version (7.4.x) handling it.
This tool would be an extension that the administrator can install and execute *after* the XWiki upgrade have been done. After its execution, users should be able to create children to every existing pages (that was previously terminal, obviously). But the old hierarchy based on the parent/child relationship, and all the preferences and rights, should be preserved. The URL should not be broken neither.
I have started a design document with the use-cases I had in mind. I've also suggested some implementations and drew some mock-ups. The results is here:
http://design.xwiki.org/xwiki/bin/view/Proposal/UpgradeToNestedPages
I've started this document before talking here in order to not come
with
nothing, but feel free to add other use-cases, ideas, and edge-cases that I haven't listed. We should also identify any blocking point from XWiki Enterprise that we could fix in a 7.4.x version.
Do you have any question in mind that we should discuss here? Do you have an opinion on this?
You help is warmly welcome,
Thanks, -- Guillaume Delhumeau (gdelhumeau@xwiki.com) Research & Development Engineer at XWiki SAS Committer on the XWiki.org project _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Guillaume Delhumeau (gdelhumeau@xwiki.com) Research & Development Engineer at XWiki SAS Committer on the XWiki.org project _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Guillaume Delhumeau (gdelhumeau@xwiki.com) Research & Development Engineer at XWiki SAS Committer on the XWiki.org project _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
Done: http://markmail.org/message/3f5b4sdu5zxpry53 Initially, I wanted to post this brainstorm on the users mailing list too, but I was afraid it would be too technical at first. Thanks, 2016-01-19 18:53 GMT+01:00 vincent@massol.net <vincent@massol.net>:
Cool Guillaume, maybe you should send that mail to the user list since we could get feedback from users trying to migrate to 7.4?
Thanks -Vincent On 19 Jan 2016 at 18:34:39, Guillaume Louis-Marie Delhumeau ( gdelhumeau@xwiki.com) wrote:
Hi.
I have released a first alpha version of the application:
http://extensions.xwiki.org/xwiki/bin/view/Extension/Nested+Pages+Migrator+A...
This application allows to compute a plan of actions to do to perform a migration from terminal pages to nested pages. The current version does not propose to apply this plan so it is only available for testing purpose.
Current limitations:
- The plan is not applied yet. - The preferences and the configured rights are not converted yet. - 2 pages can have the same target location. - There is no real pagination
You should test it on a wiki containing a lot of documents. You need to upgrade your wiki to a version handling Nested Pages before using the application.
Sometimes, the parent/based relationship between pages is messy. For example, I have run this script before using my tool on a local copy of design.xwiki.org because a lot of pages had Main.WebHome as parent meanwhile it was not justified:
{{velocity}} #set ($xwql = "where doc.space in ('Proposal', 'Design', 'Improvements') and doc.parent = 'Main.WebHome'") #foreach($r in $services.query.xwql($xwql).execute()) * $r #set ($d = $xwiki.getDocument($r)) #set ($discard = $d.setParent('Proposal.WebHome')) #set ($discard = $d.save()) #end {{/velocity}}
The UI is a single-page application that handles some options that I let you discover.
I hope you will have some time to test it and that you will like the tool. Thanks for your feedbacks,
Guillaume
2016-01-13 12:54 GMT+01:00 Ecaterina Moraru (Valica) <valicac@gmail.com>:
Hi Guillaume,
Thanks for your answer.
9. Proposing to create a backup depends on how the user will start the migration tool. If the migration is automatic, than we should create an automated backup. If it's on demand, then we need to suggest to the user to create one before.
12. The issue is for the migration process to not be stuck when encountering problematic pages and at the end to provide a list of failed pages in order for the Administrator to investigate and fix them.
Thank you Guillaume, Caty
On Wed, Jan 13, 2016 at 1:05 PM, Guillaume "Louis-Marie" Delhumeau < gdelhumeau@xwiki.com> wrote:
Hi Caty and thanks for your detailed answer. design.xwiki.org seems to be a perfect sample to test and develop this migration tool.
Now, let's see point by point.
1. The idea is to have breadcrumbs perfectly identical (ISO) after the migration. However, I was just considering moving the pages from a location to an other, not completely renaming them. I think renaming using the title (which could be velocity code in some places) is out of the scope and could be covered by... an other refactoring tool!
2. Back-links (in the content) will be updated like it usually happens when we rename a page from the UI. We should make sure the tool use the existing mechanism. However, I'm quite sure it won't affect objects properties.
3. Bookmarks won't be broken if we add a redirection to the new page at the old location. Fortunately, Marius have added this feature in 7.4: http://jira.xwiki.org/browse/XWIKI-3622
4. Yes, converting terminal pages to nested pages in the core principle of this tool. It gives the ability to create children to any existing page. Of course, this change should not be done on technical documents, and even in data generated by applications. Otherwise, this application could be broken.
5. I need to see if image references are affected by the backlinks update feature we already have. Same for macro parameters.
6. URL will be long, yes. I don't think URL aliases are planned for 8.0 and it's sure that we won't have them in 7.4.x.
8. Your queries should continue to work, since I propose to update the "parent" field when the parent is renamed. Nested pages will still have the same parent. However, it would be safer to use a query looking at the nested children similar to the one of the children viewer.
9. IMO, we could only propose to generate a full back-up XAR (with history, author preserved, etc...) but it's not the scope of this tool to manage backups.
10. I think it's ok to not handle documents from the recycle bin. Moreover, it will be still possible to run again the tool after some pages have been restored so they eventually reach their ideal location.
11. Adding a white list of spaces to convert could be a great option. For technical documents, for need to create filters. For example: do not convert hidden pages, or pages having some objects of some classes.
12. Moving pages holding a lot of attachments is a problem. See: http://jira.xwiki.org/browse/XWIKI-8910
This tool won't fix bugs that we have for ages. It means we should report any failures during the process in a list that we show to the user. I'm afraid the perfect tool cannot be made and users will still have manual things to do - but at least this tool will make them saving time.
Is that OK for you?
Any other opinion?
Thanks, Guillaume
2016-01-12 18:30 GMT+01:00 Ecaterina Moraru (Valica) < valicac@gmail.com :
12. When I moved the pages from incubator to design.xwiki.org I mostly used scripts. But I had some problems for pages that contain lots of attachments, like
http://design.xwiki.org/xwiki/bin/view/Improvements/Skin4x#Attachments
Those pages I needed to import with Large import or something, and some even recreated manually. To be honest I don't quite remember, but just be careful that the migrator needs to consider cases when pages cannot be renamed / deleted / etc.
Thanks, Caty
On Tue, Jan 12, 2016 at 7:26 PM, Ecaterina Moraru (Valica) < valicac@gmail.com> wrote:
Hi,
The following ideas are about the Parent/Children relation (UC1):
For example a copy of design.xwiki.org could be used as a test example for this kind of migration. design.xwiki.org is using a very basic application
http://extensions.xwiki.org/xwiki/bin/view/Extension/Proposal+Application
that at its core uses the Parent/Child relation in order to list in a livetable the children of a particular proposal.
The hierarchy contains pages from 2 spaces: Improvements (from the old application from incubator.myxwiki.org) and Proposal.
Sometimes the hierarchy can go to more than 6 levels, examples:
http://design.xwiki.org/xwiki/bin/view/Proposal/FlamingoAddMenuLocationItera...
or http://design.xwiki.org/xwiki/bin/view/Proposal/XWiki+Icon+Set
But sometimes the hierarchy mixes pages from the different spaces, example: http://design.xwiki.org/xwiki/bin/view/Proposal/PanelsImprovements (where Flamingo comes from Improvements space, while the others are from Proposal space).
User expectations: 1. I would really like that after the migration the hierarchy to remain the same a.k.a the breadcrumb to be unchanged. The breadcrumb navigation stays at the core of this application. I give more importance to the Page Title than the Page Name, so I would be ok for the new pages to be copied / created using the title and discard the page name. 2. But I sometimes referenciate in the content of some proposals other pages using the page name references. So what will happen to those links? Can we auto-update them inside the content with the new pages (backlinks)? 3. in your UC7: Bookmarks should not be broken after the migration: how can we achieve this? Are the page aliases a solution? Everywhere on jira there are references with links towards the design.xwiki.org proposals. For example, when I moved the app from incubator.myxwiki.org to design.xwiki.org I replaced the content of the page with a Moved type message, example http://incubator.myxwiki.org/xwiki/bin/view/Improvements/XWikiOrg . This could be a solution if we don't provide page aliases or redirects, although might not be very efficient. 4. In order to achieve the hierarchy this means that terminal pages need to be transformed into spaces? 5. In my proposals I use the {{gallery}} macro that uses references like "image:loginDesktopSnapshotAnnotations.png" or "image:Skin4xMenus@menusMenuAnnotations.png", etc. How will the images be affected? 6. Those long URL will look kind of ugly - so again page aliases would be great. But not sure how we will manage them. Do we allow multiple aliases to be created and used, or just 1? 7. What is the complexity of the changes I need to add to the Proposal App in order to continue working as before? Do we provide such documentation for the users? 8. The livetable was using this in order to list the children #set ($discard = $options.put('extraParams', "& parent=${doc.fullName}")) and #set ($importPages = $xwiki.queryManager.xwql("where doc.parent='$doc'").execute()) 9. If the user might start and finish the migration but then he will discover that everything looks bad, what can he do? A best practice would be for him to make a backup before doing the migration, but can't we make one automatically? 10. After the migration what will happen if an user goes to the 'Recycle Bin' and restores a page from "Deleted Documents" or "Deleted Attachments"? 11. Currently there are 1246 pages on the Design wiki. Improvements space contains 317 pages, while Proposal space contains 256 pages. Apparently there are other spaces that contain user generated content like Design, Tech, Standard, etc spaces. It would be great if the migrator could list user created pages when the migrator ask what pages needs to be migrated (UC3). But that list should eliminate default pages and applications pages (technical). Also maybe the user wants to migrate the hierarchy just for some pages (just the spaces Proposal and Improvements).
I know maybe the design.xwiki.org is not the best or the most complex example, but as an user I am quite concerned with what will happen to the hierarchy we've created over the years and I'm sure every user thinks the same about their content.
Thanks, Caty
On Tue, Jan 12, 2016 at 6:37 PM, Guillaume "Louis-Marie" Delhumeau < gdelhumeau@xwiki.com> wrote:
Hello everyone.
I am working on a tool to help the user migrating from an XWiki instance without the Nested Pages feature to a new version (7.4.x) handling it.
This tool would be an extension that the administrator can install and execute *after* the XWiki upgrade have been done. After its execution, users should be able to create children to every existing pages (that was previously terminal, obviously). But the old hierarchy based on the parent/child relationship, and all the preferences and rights, should be preserved. The URL should not be broken neither.
I have started a design document with the use-cases I had in mind. I've also suggested some implementations and drew some mock-ups. The results is here:
http://design.xwiki.org/xwiki/bin/view/Proposal/UpgradeToNestedPages
I've started this document before talking here in order to not
come with
nothing, but feel free to add other use-cases, ideas, and edge-cases that I haven't listed. We should also identify any blocking point from XWiki Enterprise that we could fix in a 7.4.x version.
Do you have any question in mind that we should discuss here? Do you have an opinion on this?
You help is warmly welcome,
Thanks, -- Guillaume Delhumeau (gdelhumeau@xwiki.com) Research & Development Engineer at XWiki SAS Committer on the XWiki.org project _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Guillaume Delhumeau (gdelhumeau@xwiki.com) Research & Development Engineer at XWiki SAS Committer on the XWiki.org project _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Guillaume Delhumeau (gdelhumeau@xwiki.com) Research & Development Engineer at XWiki SAS Committer on the XWiki.org project _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Guillaume Delhumeau (gdelhumeau@xwiki.com) Research & Development Engineer at XWiki SAS Committer on the XWiki.org project
participants (3)
-
Ecaterina Moraru (Valica) -
Guillaume "Louis-Marie" Delhumeau -
vincent＠massol.net