[xwiki-devs] [Proposal] [GSOC] Wiki Importer Enhancement
Hi Devs, As the existing Wiki Importer ( GSOC PROJECT - SANDBOX <http://svn.xwiki.org/svnroot/xwiki/sandbox/gsoc/importexport/>) have few set backs in architecture, i would like to make a proposal to modify and enhance to make it better. Here is my proposal : http://gsoc.myxwiki.org/xwiki/bin/view/Import+Export+other+wikis/Wiki+Import... - A new Wiki model api ( Wiki, Wiki Page, Wiki Page Revision, Wiki Attachment ) which adheres to XWiki Standards. - Separation of Import process, Parsing the import data into a abstract wiki model will be a component. Clients can use it to make their own custom implementation. This abstract model is used by the other XWiki component / client code (Groovy & velocity ) to load the data into XWiki with standard features and little customization (like Preserving history, default names, attachments etc.. ). - Using the existing XWiki model api or xwiki-bridge rather than coming up with a new one. But the issues here is, i dont have all the things which are needed and it has lot more methods which are of no use in import process. Can i proceed with a custom wiki model like this. ? - As wiki importer process large streams of data, would it be wise to design the API more stream oriented and include few stream oriented methods.? I would like to have your suggestions on this. Thanks -Arun Reddy
On Mon, Oct 12, 2009 at 08:48, Arun Reddy <vipunreddy.n@gmail.com> wrote:
Hi Devs,
As the existing Wiki Importer ( GSOC PROJECT - SANDBOX <http://svn.xwiki.org/svnroot/xwiki/sandbox/gsoc/importexport/>) have few set backs in architecture, i would like to make a proposal to modify and enhance to make it better.
Great !
Here is my proposal :
http://gsoc.myxwiki.org/xwiki/bin/view/Import+Export+other+wikis/Wiki+Import...
- A new Wiki model api ( Wiki, Wiki Page, Wiki Page Revision, Wiki Attachment ) which adheres to XWiki Standards.
- Separation of Import process, Parsing the import data into a abstract wiki model will be a component. Clients can use it to make their own custom implementation. This abstract model is used by the other XWiki component / client code (Groovy & velocity ) to load the data into XWiki with standard features and little customization (like Preserving history, default names, attachments etc.. ).
+1 Always good to separate as much as possible every aspect of the process in different components.
- Using the existing XWiki model api or xwiki-bridge rather than coming up with a new one. But the issues here is, i dont have all the things which are needed and it has lot more methods which are of no use in import process. Can i proceed with a custom wiki model like this. ?
Main problem is that you need a XDOM based model instead of a String content based model right ? The current model will definitely don't fit this, too bad the new model is not advanced enough. Also the current model is too linked to the storage so it will not be easy to do a clean stream based process. I would say do you own clean custom model and we will see latter for refactor with the new model.
- As wiki importer process large streams of data, would it be wise to design the API more stream oriented and include few stream oriented methods.?
Big +1 for stream oriented, the "normal" importer should be stream based also IMO. Denis Gervalle is currently working on making the whole export process and its dependencies stream oriented (See http://markmail.org/message/vzfdkteguzselouy). It's definitely the way to go. We can always to a wrapper api which is using the stream based api if we need a not stream based tools but the other way around is not possible so better to do now the more generic one.
I would like to have your suggestions on this.
Thanks -Arun Reddy _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne
On Mon, Oct 12, 2009 at 11:56, Thomas Mortagne <thomas.mortagne@xwiki.com> wrote:
On Mon, Oct 12, 2009 at 08:48, Arun Reddy <vipunreddy.n@gmail.com> wrote:
Hi Devs,
As the existing Wiki Importer ( GSOC PROJECT - SANDBOX <http://svn.xwiki.org/svnroot/xwiki/sandbox/gsoc/importexport/>) have few set backs in architecture, i would like to make a proposal to modify and enhance to make it better.
Great !
Here is my proposal :
http://gsoc.myxwiki.org/xwiki/bin/view/Import+Export+other+wikis/Wiki+Import...
- A new Wiki model api ( Wiki, Wiki Page, Wiki Page Revision, Wiki Attachment ) which adheres to XWiki Standards.
Here is some quick review comments: General: * this API is not streaming oriented, WikiImporter.import should take listener and not send a full Wiki for example. This api is nice but as a wrapper for the streaming api (like SAX and DOM or XWiki rednering events and XDOM). * Wiki Importer ** you should add some more informationnal methods to use in the UI or to make easier to do a generic tool like: *** String getName() : the pretty name of the importer *** String getDescriptions(): a description of the importer *** [NICE TO HAVE] List<Class<?>> getParameters(): the list of parameters type to provide to the importer, that way you can generate a UI based on it since parameters totally depends on the importer, you could have an importer able to import the wiki from an URL for example using REST/XMLRPC... for the same reason List<InputStream> is not generic enough IMO. You can look at MacroDescriptor for inspiration, we could make it more generic as a list of parameters instead of a list of macro parameters to use it in wiki importer, authenticators, etc. I see this tool as a generic way to get wiki content from any kind of external source, we could even use it to copy a XWiki wiki without having to do an export/import by hand for example.
- Separation of Import process, Parsing the import data into a abstract wiki model will be a component. Clients can use it to make their own custom implementation. This abstract model is used by the other XWiki component / client code (Groovy & velocity ) to load the data into XWiki with standard features and little customization (like Preserving history, default names, attachments etc.. ).
+1
Always good to separate as much as possible every aspect of the process in different components.
- Using the existing XWiki model api or xwiki-bridge rather than coming up with a new one. But the issues here is, i dont have all the things which are needed and it has lot more methods which are of no use in import process. Can i proceed with a custom wiki model like this. ?
Main problem is that you need a XDOM based model instead of a String content based model right ? The current model will definitely don't fit this, too bad the new model is not advanced enough. Also the current model is too linked to the storage so it will not be easy to do a clean stream based process.
I would say do you own clean custom model and we will see latter for refactor with the new model.
- As wiki importer process large streams of data, would it be wise to design the API more stream oriented and include few stream oriented methods.?
Big +1 for stream oriented, the "normal" importer should be stream based also IMO. Denis Gervalle is currently working on making the whole export process and its dependencies stream oriented (See http://markmail.org/message/vzfdkteguzselouy).
It's definitely the way to go. We can always to a wrapper api which is using the stream based api if we need a not stream based tools but the other way around is not possible so better to do now the more generic one.
I would like to have your suggestions on this.
Thanks -Arun Reddy _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne
-- Thomas Mortagne
Hi, Here is some quick review comments:
General: * this API is not streaming oriented, WikiImporter.import should take listener and not send a full Wiki for example. This api is nice but as a wrapper for the streaming api (like SAX and DOM or XWiki rednering events and XDOM).
I kind of had a different idea about the streaming api. I don't know why I thought this but what I had in mind was something like: Wiki::hasNextPage():boolean Wiki::getNextPage():WikiPage This is kind of like reading a result set (pull-parsing???) I thought this kind of a api would be more natural but I have no idea if this is even possible to implement for all the wiki export formats out there. If I understood you correct, what you have suggested is something like: WikiImporterListener::onWikiPage(WikiPage) WikiImporter::import(......, WikiImporterListener) This might be viewed as a "push-parsing" mechanism?? Anyway, I'll be glad to know what are the pros & cons of these two approaches.
*** [NICE TO HAVE] List<Class<?>> getParameters(): the list of parameters type to provide to the importer, that way you can generate a UI based on it since parameters totally depends on the importer, you could have an importer able to import the wiki from an URL for example using REST/XMLRPC... for the same reason List<InputStream> is not generic enough IMO. You can look at MacroDescriptor for inspiration, we could make it more generic as a list of parameters instead of a list of macro parameters to use it in wiki importer, authenticators, etc. I see this tool as a generic way to get wiki content from any kind of external source, we could even use it to copy a XWiki wiki without having to do an export/import by hand for example.
+1 This will make it possible to define different types of importers with different requirements (rather than restricting them to List<InputStream>) Thanks. - Asiri
On Mon, Oct 12, 2009 at 17:57, Asiri Rathnayake <asiri.rathnayake@gmail.com> wrote:
Hi,
Here is some quick review comments:
General: * this API is not streaming oriented, WikiImporter.import should take listener and not send a full Wiki for example. This api is nice but as a wrapper for the streaming api (like SAX and DOM or XWiki rednering events and XDOM).
I kind of had a different idea about the streaming api. I don't know why I thought this but what I had in mind was something like:
Wiki::hasNextPage():boolean
Wiki::getNextPage():WikiPage
This is kind of like reading a result set (pull-parsing???)
I thought this kind of a api would be more natural but I have no idea if this is even possible to implement for all the wiki export formats out there.
If I understood you correct, what you have suggested is something like:
WikiImporterListener::onWikiPage(WikiPage)
Not sure yet for the details but that's the idea yes.
WikiImporter::import(......, WikiImporterListener)
This might be viewed as a "push-parsing" mechanism?? Anyway, I'll be glad to know what are the pros & cons of these two approaches.
Actually in my mind there was more than just this method, i see at least also WikiImporterListener::onWikiPageAttachment(AttachmentName, InputStream) since having the whole WikiPage with all its attachments in memory make the import not strong enough if you have too big attachments. Making a streaming API is about making every memory consuming part streamed (Actually to be sure maybe also have WikiImporterListener::onWikiPageObject(WikiPageObject) but since XWiki does not support this well internally currently it's less important than attachments). The getNextPage() API is good when you have only one simple kind of data in a plane list but the events based way is more generic and easier to implements for the importer writer IMO since it send the events when he wants.
*** [NICE TO HAVE] List<Class<?>> getParameters(): the list of parameters type to provide to the importer, that way you can generate a UI based on it since parameters totally depends on the importer, you could have an importer able to import the wiki from an URL for example using REST/XMLRPC... for the same reason List<InputStream> is not generic enough IMO. You can look at MacroDescriptor for inspiration, we could make it more generic as a list of parameters instead of a list of macro parameters to use it in wiki importer, authenticators, etc. I see this tool as a generic way to get wiki content from any kind of external source, we could even use it to copy a XWiki wiki without having to do an export/import by hand for example.
+1
This will make it possible to define different types of importers with different requirements (rather than restricting them to List<InputStream>)
Thanks.
- Asiri _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne
Hi, Actually in my mind there was more than just this method, i see at least
also
WikiImporterListener::onWikiPageAttachment(AttachmentName, InputStream)
since having the whole WikiPage with all its attachments in memory make the import not strong enough if you have too big attachments. Making a streaming API is about making every memory consuming part streamed (Actually to be sure maybe also have WikiImporterListener::onWikiPageObject(WikiPageObject) but since XWiki does not support this well internally currently it's less important than attachments).
The getNextPage() API is good when you have only one simple kind of data in a plane list but the events based way is more generic and easier to implements for the importer writer IMO since it send the events when he wants.
Ok, Now I understand that importer writers work would be easy if we go with an event based API. So for Arun this means, 1. The model you proposed is ok 2. You need to refine the WikiImporter API so that it takes into account what we discussed here. This means: i) Have a mechanism for defining WikiImporter components with different requirements (look at the getParameters() approach suggested by Thomas) ii) Make the WikiImporter api mode stream oriented. I think you can start with onWikiPage() & onAttachment() events for now. I think you should start working on a revised proposal with this information taken into account. In the mean time other developers will comment on this proposal if they have further suggestions. - Asiri
*** [NICE TO HAVE] List<Class<?>> getParameters(): the list of parameters type to provide to the importer, that way you can generate a UI based on it since parameters totally depends on the importer, you could have an importer able to import the wiki from an URL for example using REST/XMLRPC... for the same reason List<InputStream> is not generic enough IMO. You can look at MacroDescriptor for inspiration, we could make it more generic as a list of parameters instead of a list of macro parameters to use it in wiki importer, authenticators, etc. I see this tool as a generic way to get wiki content from any kind of external source, we could even use it to copy a XWiki wiki without having to do an export/import by hand for example.
+1
This will make it possible to define different types of importers with different requirements (rather than restricting them to
List<InputStream>)
Thanks.
- Asiri _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
Hi, On Mon, Oct 12, 2009 at 10:27 PM, Asiri Rathnayake < asiri.rathnayake@gmail.com> wrote:
Hi,
Actually in my mind there was more than just this method, i see at least
also
WikiImporterListener::onWikiPageAttachment(AttachmentName, InputStream)
since having the whole WikiPage with all its attachments in memory make the import not strong enough if you have too big attachments. Making a streaming API is about making every memory consuming part streamed (Actually to be sure maybe also have WikiImporterListener::onWikiPageObject(WikiPageObject) but since XWiki does not support this well internally currently it's less important than attachments).
The getNextPage() API is good when you have only one simple kind of data in a plane list but the events based way is more generic and easier to implements for the importer writer IMO since it send the events when he wants.
Ok, Now I understand that importer writers work would be easy if we go with an event based API.
So for Arun this means,
1. The model you proposed is ok
2. You need to refine the WikiImporter API so that it takes into account what we discussed here. This means:
i) Have a mechanism for defining WikiImporter components with different requirements (look at the getParameters() approach suggested by Thomas)
ii) Make the WikiImporter api mode stream oriented. I think you can start with onWikiPage() & onAttachment() events for now.
- Can we have beginWikiPage() followed by endWikiPage() beginAttachment() and endAttachment() rather than having a single onWikiPage(),onAttachment() in the WikiImporterListener. -How to handle revisions - Can we have methods to read those tags too. one for WikiPageRevision and other for AttachmentRevision.
I think you should start working on a revised proposal with this information taken into account. In the mean time other developers will comment on this proposal if they have further suggestions.
- Asiri
*** [NICE TO HAVE] List<Class<?>> getParameters(): the list of parameters type to provide to the importer, that way you can generate a UI based on it since parameters totally depends on the importer, you could have an importer able to import the wiki from an URL for example using REST/XMLRPC... for the same reason List<InputStream> is not generic enough IMO. You can look at MacroDescriptor for inspiration, we could make it more generic as a list of parameters instead of a list of macro parameters to use it in wiki importer, authenticators, etc. I see this tool as a generic way to get wiki content from any kind of external source, we could even use it to copy a XWiki wiki without having to do an export/import by hand for example.
+1
This will make it possible to define different types of importers with different requirements (rather than restricting them to
List<InputStream>)
Thanks.
- Asiri _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Best Regards, Arun Reddy
On Wed, Oct 14, 2009 at 17:03, Arun Reddy <vipunreddy.n@gmail.com> wrote:
Hi,
On Mon, Oct 12, 2009 at 10:27 PM, Asiri Rathnayake < asiri.rathnayake@gmail.com> wrote:
Hi,
Actually in my mind there was more than just this method, i see at least
also
WikiImporterListener::onWikiPageAttachment(AttachmentName, InputStream)
since having the whole WikiPage with all its attachments in memory make the import not strong enough if you have too big attachments. Making a streaming API is about making every memory consuming part streamed (Actually to be sure maybe also have WikiImporterListener::onWikiPageObject(WikiPageObject) but since XWiki does not support this well internally currently it's less important than attachments).
The getNextPage() API is good when you have only one simple kind of data in a plane list but the events based way is more generic and easier to implements for the importer writer IMO since it send the events when he wants.
Ok, Now I understand that importer writers work would be easy if we go with an event based API.
So for Arun this means,
1. The model you proposed is ok
2. You need to refine the WikiImporter API so that it takes into account what we discussed here. This means:
i) Have a mechanism for defining WikiImporter components with different requirements (look at the getParameters() approach suggested by Thomas)
ii) Make the WikiImporter api mode stream oriented. I think you can start with onWikiPage() & onAttachment() events for now.
- Can we have beginWikiPage() followed by endWikiPage() beginAttachment() and endAttachment() rather than having a single onWikiPage(),onAttachment() in the WikiImporterListener.
If you plan to have something like beginWikiPage(name) beginObject(type) onProperty(name, value, ...) endObject() beginObject() onProperty() endObject() beginAttachment(name) endAttachment() endWikiPage() +1 it's a good idea, now i'm not sure why you need begin and end for an attachment, what other events would you have inside it ?
-How to handle revisions - Can we have methods to read those tags too. one for WikiPageRevision and other for AttachmentRevision.
I think you should start working on a revised proposal with this information taken into account. In the mean time other developers will comment on this proposal if they have further suggestions.
- Asiri
*** [NICE TO HAVE] List<Class<?>> getParameters(): the list of parameters type to provide to the importer, that way you can generate a UI based on it since parameters totally depends on the importer, you could have an importer able to import the wiki from an URL for example using REST/XMLRPC... for the same reason List<InputStream> is not generic enough IMO. You can look at MacroDescriptor for inspiration, we could make it more generic as a list of parameters instead of a list of macro parameters to use it in wiki importer, authenticators, etc. I see this tool as a generic way to get wiki content from any kind of external source, we could even use it to copy a XWiki wiki without having to do an export/import by hand for example.
+1
This will make it possible to define different types of importers with different requirements (rather than restricting them to
List<InputStream>)
Thanks.
- Asiri _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Best Regards, Arun Reddy _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne
On Wed, Oct 14, 2009 at 8:41 PM, Thomas Mortagne <thomas.mortagne@xwiki.com>wrote:
On Wed, Oct 14, 2009 at 17:03, Arun Reddy <vipunreddy.n@gmail.com> wrote:
Hi,
On Mon, Oct 12, 2009 at 10:27 PM, Asiri Rathnayake < asiri.rathnayake@gmail.com> wrote:
Hi,
Actually in my mind there was more than just this method, i see at least
also
WikiImporterListener::onWikiPageAttachment(AttachmentName, InputStream)
since having the whole WikiPage with all its attachments in memory make the import not strong enough if you have too big attachments. Making a streaming API is about making every memory consuming part streamed (Actually to be sure maybe also have WikiImporterListener::onWikiPageObject(WikiPageObject) but since XWiki does not support this well internally currently it's less important than attachments).
The getNextPage() API is good when you have only one simple kind of data in a plane list but the events based way is more generic and easier to implements for the importer writer IMO since it send the events when he wants.
Ok, Now I understand that importer writers work would be easy if we go with an event based API.
So for Arun this means,
1. The model you proposed is ok
2. You need to refine the WikiImporter API so that it takes into account what we discussed here. This means:
i) Have a mechanism for defining WikiImporter components with different requirements (look at the getParameters() approach suggested by Thomas)
ii) Make the WikiImporter api mode stream oriented. I think you can start with onWikiPage() & onAttachment() events for now.
- Can we have beginWikiPage() followed by endWikiPage() beginAttachment() and endAttachment() rather than having a single onWikiPage(),onAttachment() in the WikiImporterListener.
If you plan to have something like
beginWikiPage(name) beginObject(type) onProperty(name, value, ...) endObject() beginObject() onProperty() endObject() beginAttachment(name) endAttachment() endWikiPage()
+1 it's a good idea, now i'm not sure why you need begin and end for an attachment, what other events would you have inside it ?
Because even it has few properties and revisions. Kindly correct me, if am wrong.
-How to handle revisions - Can we have methods to read those tags too.
one
for WikiPageRevision and other for AttachmentRevision.
I think you should start working on a revised proposal with this information taken into account. In the mean time other developers will comment on this proposal if they have further suggestions.
- Asiri
*** [NICE TO HAVE] List<Class<?>> getParameters(): the list of parameters type to provide to the importer, that way you can
generate
a UI based on it since parameters totally depends on the importer, you could have an importer able to import the wiki from an URL for example using REST/XMLRPC... for the same reason List<InputStream> is not generic enough IMO. You can look at MacroDescriptor for inspiration, we could make it more generic as a list of parameters instead of a list of macro parameters to use it in wiki importer, authenticators, etc. I see this tool as a generic way to get wiki content from any kind of external source, we could even use it to copy a XWiki wiki without having to do an export/import by hand for example.
+1
This will make it possible to define different types of importers with different requirements (rather than restricting them to List<InputStream>)
Thanks.
- Asiri _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Best Regards, Arun Reddy _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Best Regards, Arun Reddy
On Wed, Oct 14, 2009 at 17:22, Arun Reddy <vipunreddy.n@gmail.com> wrote:
On Wed, Oct 14, 2009 at 8:41 PM, Thomas Mortagne <thomas.mortagne@xwiki.com>wrote:
On Wed, Oct 14, 2009 at 17:03, Arun Reddy <vipunreddy.n@gmail.com> wrote:
Hi,
On Mon, Oct 12, 2009 at 10:27 PM, Asiri Rathnayake < asiri.rathnayake@gmail.com> wrote:
Hi,
Actually in my mind there was more than just this method, i see at least
also
WikiImporterListener::onWikiPageAttachment(AttachmentName, InputStream)
since having the whole WikiPage with all its attachments in memory make the import not strong enough if you have too big attachments. Making a streaming API is about making every memory consuming part streamed (Actually to be sure maybe also have WikiImporterListener::onWikiPageObject(WikiPageObject) but since XWiki does not support this well internally currently it's less important than attachments).
The getNextPage() API is good when you have only one simple kind of data in a plane list but the events based way is more generic and easier to implements for the importer writer IMO since it send the events when he wants.
Ok, Now I understand that importer writers work would be easy if we go with an event based API.
So for Arun this means,
1. The model you proposed is ok
2. You need to refine the WikiImporter API so that it takes into account what we discussed here. This means:
i) Have a mechanism for defining WikiImporter components with different requirements (look at the getParameters() approach suggested by Thomas)
ii) Make the WikiImporter api mode stream oriented. I think you can start with onWikiPage() & onAttachment() events for now.
- Can we have beginWikiPage() followed by endWikiPage() beginAttachment() and endAttachment() rather than having a single onWikiPage(),onAttachment() in the WikiImporterListener.
If you plan to have something like
beginWikiPage(name) beginObject(type) onProperty(name, value, ...) endObject() beginObject() onProperty() endObject() beginAttachment(name) endAttachment() endWikiPage()
+1 it's a good idea, now i'm not sure why you need begin and end for an attachment, what other events would you have inside it ?
Because even it has few properties and revisions. Kindly correct me, if am wrong.
You don't need an event for each property, event are more for variables things, you can have onAttachment(name, version, content, etc.). For revisions i already anwser in the previous mail, if you put the onAttachement inside a begin/endWikiPageRevision you will only have one version of the attachement for a version of wiki page.
>
-How to handle revisions - Can we have methods to read those tags too. one for WikiPageRevision and other for AttachmentRevision.
I think you should start working on a revised proposal with this information taken into account. In the mean time other developers will comment on this proposal if they have further suggestions.
- Asiri
> *** [NICE TO HAVE] List<Class<?>> getParameters(): the list of > parameters type to provide to the importer, that way you can
generate
> a UI based on it since parameters totally depends on the importer, you > could have an importer able to import the wiki from an URL for example > using REST/XMLRPC... for the same reason List<InputStream> is not > generic enough IMO. You can look at MacroDescriptor for inspiration, > we could make it more generic as a list of parameters instead of a > list of macro parameters to use it in wiki importer, authenticators, > etc. I see this tool as a generic way to get wiki content from any > kind of external source, we could even use it to copy a XWiki wiki > without having to do an export/import by hand for example. >
+1
This will make it possible to define different types of importers with different requirements (rather than restricting them to List<InputStream>)
Thanks.
- Asiri _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Best Regards, Arun Reddy _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Best Regards, Arun Reddy _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne
On Wed, Oct 14, 2009 at 17:11, Thomas Mortagne <thomas.mortagne@xwiki.com> wrote:
On Wed, Oct 14, 2009 at 17:03, Arun Reddy <vipunreddy.n@gmail.com> wrote:
Hi,
On Mon, Oct 12, 2009 at 10:27 PM, Asiri Rathnayake < asiri.rathnayake@gmail.com> wrote:
Hi,
Actually in my mind there was more than just this method, i see at least
also
WikiImporterListener::onWikiPageAttachment(AttachmentName, InputStream)
since having the whole WikiPage with all its attachments in memory make the import not strong enough if you have too big attachments. Making a streaming API is about making every memory consuming part streamed (Actually to be sure maybe also have WikiImporterListener::onWikiPageObject(WikiPageObject) but since XWiki does not support this well internally currently it's less important than attachments).
The getNextPage() API is good when you have only one simple kind of data in a plane list but the events based way is more generic and easier to implements for the importer writer IMO since it send the events when he wants.
Ok, Now I understand that importer writers work would be easy if we go with an event based API.
So for Arun this means,
1. The model you proposed is ok
2. You need to refine the WikiImporter API so that it takes into account what we discussed here. This means:
i) Have a mechanism for defining WikiImporter components with different requirements (look at the getParameters() approach suggested by Thomas)
ii) Make the WikiImporter api mode stream oriented. I think you can start with onWikiPage() & onAttachment() events for now.
- Can we have beginWikiPage() followed by endWikiPage() beginAttachment() and endAttachment() rather than having a single onWikiPage(),onAttachment() in the WikiImporterListener.
If you plan to have something like
beginWikiPage(name) beginObject(type) onProperty(name, value, ...) endObject() beginObject() onProperty() endObject() beginAttachment(name) endAttachment() endWikiPage()
+1 it's a good idea, now i'm not sure why you need begin and end for an attachment, what other events would you have inside it ?
-How to handle revisions - Can we have methods to read those tags too. one for WikiPageRevision and other for AttachmentRevision.
Maybe something like: beginWikiPage() beginWikiPageRevision() beginObject(type) onProperty() endObject() beginObject() onProperty() endObject() beginClass() onProperty() endClass() beginAttachment() onAttachmentRevision() endAttachment() endWikiPage() beginWikiPageRevision() beginObject(type) onProperty() endObject() beginObject() onProperty() endObject() onAttachmentRevision() (since i think you can't have several attachement revision for the same wiki page revision) endWikiPage() endWikiPage() basically you can look at XWiki xml ecport format and map something similar for events I think (not the exact same thing the export format need improvements).
I think you should start working on a revised proposal with this information taken into account. In the mean time other developers will comment on this proposal if they have further suggestions.
- Asiri
*** [NICE TO HAVE] List<Class<?>> getParameters(): the list of parameters type to provide to the importer, that way you can generate a UI based on it since parameters totally depends on the importer, you could have an importer able to import the wiki from an URL for example using REST/XMLRPC... for the same reason List<InputStream> is not generic enough IMO. You can look at MacroDescriptor for inspiration, we could make it more generic as a list of parameters instead of a list of macro parameters to use it in wiki importer, authenticators, etc. I see this tool as a generic way to get wiki content from any kind of external source, we could even use it to copy a XWiki wiki without having to do an export/import by hand for example.
+1
This will make it possible to define different types of importers with different requirements (rather than restricting them to
List<InputStream>)
Thanks.
- Asiri _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Best Regards, Arun Reddy _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne
-- Thomas Mortagne
Hi,
-How to handle revisions - Can we have methods to read those tags too. one
for WikiPageRevision and other for AttachmentRevision.
Maybe something like:
beginWikiPage() beginWikiPageRevision() beginObject(type) onProperty() endObject() beginObject() onProperty() endObject() beginClass() onProperty() endClass() beginAttachment() onAttachmentRevision() endAttachment() endWikiPage() beginWikiPageRevision() beginObject(type) onProperty() endObject() beginObject() onProperty() endObject() onAttachmentRevision() (since i think you can't have several attachement revision for the same wiki page revision) endWikiPage() endWikiPage()
+1 for something like this. Now I don't think that other wiki formats (mediawiki, confluence) will normally fire events like beginObject() / endObject() but it's important to have the full span of XWiki's capabilities embodied into this API. Because then we can, 1) Use this generic importer even when importing xwiki xml exports, 2) We'll be able to map different concepts of other wiki exports into xwiki concepts. An example is mapping [category] links of mediawiki into xwiki tags. Thanks. - Asiri
basically you can look at XWiki xml ecport format and map something similar for events I think (not the exact same thing the export format need improvements).
I think you should start working on a revised proposal with this information taken into account. In the mean time other developers will comment on
this
proposal if they have further suggestions.
- Asiri
> *** [NICE TO HAVE] List<Class<?>> getParameters(): the list of > parameters type to provide to the importer, that way you can
generate
> a UI based on it since parameters totally depends on the importer, you > could have an importer able to import the wiki from an URL for example > using REST/XMLRPC... for the same reason List<InputStream> is not > generic enough IMO. You can look at MacroDescriptor for inspiration, > we could make it more generic as a list of parameters instead of a > list of macro parameters to use it in wiki importer, authenticators, > etc. I see this tool as a generic way to get wiki content from any > kind of external source, we could even use it to copy a XWiki wiki > without having to do an export/import by hand for example. >
+1
This will make it possible to define different types of importers with different requirements (rather than restricting them to List<InputStream>)
Thanks.
- Asiri _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Best Regards, Arun Reddy _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne
-- Thomas Mortagne _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
On Wed, Oct 14, 2009 at 20:49, Asiri Rathnayake <asiri.rathnayake@gmail.com> wrote:
Hi,
-How to handle revisions - Can we have methods to read those tags too. one
for WikiPageRevision and other for AttachmentRevision.
Maybe something like:
beginWikiPage() beginWikiPageRevision() beginObject(type) onProperty() endObject() beginObject() onProperty() endObject() beginClass() onProperty() endClass() beginAttachment() onAttachmentRevision() endAttachment() endWikiPage() beginWikiPageRevision() beginObject(type) onProperty() endObject() beginObject() onProperty() endObject() onAttachmentRevision() (since i think you can't have several attachement revision for the same wiki page revision) endWikiPage() endWikiPage()
+1 for something like this.
Now I don't think that other wiki formats (mediawiki, confluence) will normally fire events like beginObject() / endObject() but it's important to have the full span of XWiki's capabilities embodied into this API. Because then we can,
1) Use this generic importer even when importing xwiki xml exports,
2) We'll be able to map different concepts of other wiki exports into xwiki concepts. An example is mapping [category] links of mediawiki into xwiki tags.
3) actually the use case I prefer is the possibility to copy a XWiki wiki/space on the fly using a XMLRPC or REST based importer :)
Thanks.
- Asiri
basically you can look at XWiki xml ecport format and map something similar for events I think (not the exact same thing the export format need improvements).
I think you should start working on a revised proposal with this information taken into account. In the mean time other developers will comment on
this
proposal if they have further suggestions.
- Asiri
> > >> *** [NICE TO HAVE] List<Class<?>> getParameters(): the list of >> parameters type to provide to the importer, that way you can generate >> a UI based on it since parameters totally depends on the importer, you >> could have an importer able to import the wiki from an URL for example >> using REST/XMLRPC... for the same reason List<InputStream> is not >> generic enough IMO. You can look at MacroDescriptor for inspiration, >> we could make it more generic as a list of parameters instead of a >> list of macro parameters to use it in wiki importer, authenticators, >> etc. I see this tool as a generic way to get wiki content from any >> kind of external source, we could even use it to copy a XWiki wiki >> without having to do an export/import by hand for example. >> > > +1 > > This will make it possible to define different types of importers with > different requirements (rather than restricting them to List<InputStream>) > > Thanks. > > - Asiri > _______________________________________________ > devs mailing list > devs@xwiki.org > http://lists.xwiki.org/mailman/listinfo/devs >
-- Thomas Mortagne _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Best Regards, Arun Reddy _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne
-- Thomas Mortagne _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne
Hi Thomas, On Mon, Oct 12, 2009 at 8:50 PM, Thomas Mortagne <thomas.mortagne@xwiki.com>wrote:
On Mon, Oct 12, 2009 at 11:56, Thomas Mortagne <thomas.mortagne@xwiki.com> wrote:
On Mon, Oct 12, 2009 at 08:48, Arun Reddy <vipunreddy.n@gmail.com> wrote:
Hi Devs,
As the existing Wiki Importer ( GSOC PROJECT - SANDBOX <http://svn.xwiki.org/svnroot/xwiki/sandbox/gsoc/importexport/>) have few set backs in architecture, i would like to make a proposal to modify and enhance to make it better.
Great !
Here is my proposal :
http://gsoc.myxwiki.org/xwiki/bin/view/Import+Export+other+wikis/Wiki+Import...
- A new Wiki model api ( Wiki, Wiki Page, Wiki Page Revision, Wiki Attachment ) which adheres to XWiki Standards.
Here is some quick review comments: General: * this API is not streaming oriented, WikiImporter.import should take listener and not send a full Wiki for example. This api is nice but as a wrapper for the streaming api (like SAX and DOM or XWiki rednering events and XDOM).
* Wiki Importer ** you should add some more informationnal methods to use in the UI or to make easier to do a generic tool like: *** String getName() : the pretty name of the importer *** String getDescriptions(): a description of the importer *** [NICE TO HAVE] List<Class<?>> getParameters(): the
Can we have a Descriptor class for these instead of having these methods inside WikiImporter. Something similiar to MacroDescriptor we can have WikiDescriptor. WikiDescriptor::getName():String WikiDescriptor::getDescription():String WikiDescriptor::getParametersBeanClass():Class<?> list of
parameters type to provide to the importer, that way you can generate a UI based on it since parameters totally depends on the importer, you could have an importer able to import the wiki from an URL for example using REST/XMLRPC... for the same reason List<InputStream> is not generic enough IMO. You can look at MacroDescriptor for inspiration, we could make it more generic as a list of parameters instead of a list of macro parameters to use it in wiki importer, authenticators, etc. I see this tool as a generic way to get wiki content from any kind of external source, we could even use it to copy a XWiki wiki without having to do an export/import by hand for example.
- Separation of Import process, Parsing the import data into a abstract
wiki
model will be a component. Clients can use it to make their own custom implementation. This abstract model is used by the other XWiki component / client code (Groovy & velocity ) to load the data into XWiki with standard features and little customization (like Preserving history, default names, attachments etc.. ).
+1
Always good to separate as much as possible every aspect of the process in different components.
- Using the existing XWiki model api or xwiki-bridge rather than coming
up
with a new one. But the issues here is, i dont have all the things which are needed and it has lot more methods which are of no use in import process. Can i proceed with a custom wiki model like this. ?
Main problem is that you need a XDOM based model instead of a String content based model right ? The current model will definitely don't fit this, too bad the new model is not advanced enough. Also the current model is too linked to the storage so it will not be easy to do a clean stream based process.
I would say do you own clean custom model and we will see latter for refactor with the new model.
- As wiki importer process large streams of data, would it be wise to
design
the API more stream oriented and include few stream oriented methods.?
Big +1 for stream oriented, the "normal" importer should be stream based also IMO. Denis Gervalle is currently working on making the whole export process and its dependencies stream oriented (See http://markmail.org/message/vzfdkteguzselouy).
It's definitely the way to go. We can always to a wrapper api which is using the stream based api if we need a not stream based tools but the other way around is not possible so better to do now the more generic one.
I would like to have your suggestions on this.
Thanks -Arun Reddy _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne
-- Thomas Mortagne _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Best Regards, Arun Reddy
On Thu, Nov 5, 2009 at 06:43, Arun Reddy <vipunreddy.n@gmail.com> wrote:
Hi Thomas,
On Mon, Oct 12, 2009 at 8:50 PM, Thomas Mortagne <thomas.mortagne@xwiki.com>wrote:
On Mon, Oct 12, 2009 at 11:56, Thomas Mortagne <thomas.mortagne@xwiki.com> wrote:
On Mon, Oct 12, 2009 at 08:48, Arun Reddy <vipunreddy.n@gmail.com> wrote:
Hi Devs,
As the existing Wiki Importer ( GSOC PROJECT - SANDBOX <http://svn.xwiki.org/svnroot/xwiki/sandbox/gsoc/importexport/>) have few set backs in architecture, i would like to make a proposal to modify and enhance to make it better.
Great !
Here is my proposal :
http://gsoc.myxwiki.org/xwiki/bin/view/Import+Export+other+wikis/Wiki+Import...
- A new Wiki model api ( Wiki, Wiki Page, Wiki Page Revision, Wiki Attachment ) which adheres to XWiki Standards.
Here is some quick review comments: General: * this API is not streaming oriented, WikiImporter.import should take listener and not send a full Wiki for example. This api is nice but as a wrapper for the streaming api (like SAX and DOM or XWiki rednering events and XDOM).
* Wiki Importer ** you should add some more informationnal methods to use in the UI or to make easier to do a generic tool like: *** String getName() : the pretty name of the importer *** String getDescriptions(): a description of the importer *** [NICE TO HAVE] List<Class<?>> getParameters(): the
Can we have a Descriptor class for these instead of having these methods inside WikiImporter. Something similiar to MacroDescriptor we can have WikiDescriptor.
WikiDescriptor::getName():String WikiDescriptor::getDescription():String WikiDescriptor::getParametersBeanClass():Class<?>
Sure do the same than macros, it's cleaner and more consistent that way. But it should be WikiImporterDescriptor I think, it's not a descriptor for a wiki ;)
list of
parameters type to provide to the importer, that way you can generate a UI based on it since parameters totally depends on the importer, you could have an importer able to import the wiki from an URL for example using REST/XMLRPC... for the same reason List<InputStream> is not generic enough IMO. You can look at MacroDescriptor for inspiration, we could make it more generic as a list of parameters instead of a list of macro parameters to use it in wiki importer, authenticators, etc. I see this tool as a generic way to get wiki content from any kind of external source, we could even use it to copy a XWiki wiki without having to do an export/import by hand for example.
- Separation of Import process, Parsing the import data into a abstract
wiki
model will be a component. Clients can use it to make their own custom implementation. This abstract model is used by the other XWiki component / client code (Groovy & velocity ) to load the data into XWiki with standard features and little customization (like Preserving history, default names, attachments etc.. ).
+1
Always good to separate as much as possible every aspect of the process in different components.
- Using the existing XWiki model api or xwiki-bridge rather than coming
up
with a new one. But the issues here is, i dont have all the things which are needed and it has lot more methods which are of no use in import process. Can i proceed with a custom wiki model like this. ?
Main problem is that you need a XDOM based model instead of a String content based model right ? The current model will definitely don't fit this, too bad the new model is not advanced enough. Also the current model is too linked to the storage so it will not be easy to do a clean stream based process.
I would say do you own clean custom model and we will see latter for refactor with the new model.
- As wiki importer process large streams of data, would it be wise to
design
the API more stream oriented and include few stream oriented methods.?
Big +1 for stream oriented, the "normal" importer should be stream based also IMO. Denis Gervalle is currently working on making the whole export process and its dependencies stream oriented (See http://markmail.org/message/vzfdkteguzselouy).
It's definitely the way to go. We can always to a wrapper api which is using the stream based api if we need a not stream based tools but the other way around is not possible so better to do now the more generic one.
I would like to have your suggestions on this.
Thanks -Arun Reddy _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne
-- Thomas Mortagne _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Best Regards, Arun Reddy _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne
Hi Devs, I have implemented an API for Wiki Importer. The main component is org.xwiki.wikiimporter.WikiImporter **WikiImporterDescriptor getDescriptor() **void importWiki(Object object, WikiImporterListener listener) **WikiImporterType getType() Wiki Call back event listener: org.xwiki.wikiimporter.WikiImporterListener **void beginWikiPage(String pageName, Map<String, String> params) **void beginWikiPageRevision(String pageName, int revision, Map<String, String> params) **void beginObject(String objectType, Map<String, String> params) **void onProperty(String property, Map<String, String> params, String value) **void endObject(String objectType, Map<String, String> params) **void endWikiPageRevision(String pageName, int revision, Map<String, String> params) **void endWikiPage(String pageName, Map<String, String> params) **void beginAttachment(String attachmentName, Map<String, String> param) **void onAttachmentRevision(String attachmentName, Map<String, String> params, InputSource input) **void endAttachment(String attachmentName, Map<String, String> params) WikiImporter parser for parsing documents (eg: XML ) : org.xwiki.wikiimporter.WikiImporterParser **void parse(InputSource source, WikiImporterListener listener) throws WikiImporterParseException WikiImporter velocity bridge : org.xwiki.wikiimporter.WikiImporterVelocityBridge **public WikiImporterTypeFactory getWikiImporterTypeFactory() **public WikiImporter getWikiImporter(String wikiImporterType) throws WikiImporterException WikiImporter type factory has methods to handle the supported other wiki formats/types. : org.xwiki.wikiimporter.WikiImporterTypeFactory **WikiImporterType createTypeFromIdString(String wikiImporterType) throws WikiImporterException **List<WikiImporterType> getAvailableTypes() throws WikiImporterException This is a typical use case in my mind. 1. Get WikiImporterFactory instance with WikiImporterVelocityBridge#getWikiImporterFactory 2. Call the WikiImporterTypeFactory#getAvailableTypes() --> can be used at UI level to show the supported types ( MediaWiki XML, XMLRPC etc .. ) 3. Select the format type and call WikiImporterVelocityBridge#getWikiImporter(String wikiImporterType) to get selected WikiImporter instance. 4. Use the WikiImporterDescriptor to generate the UI. ( Similiar to MacroDescriptor) 5. User enters all the parameters ( UI ) 6. Call WikiImporter#importWiki(Parameters Class Object, WikiImporterListener ) to start the import process. Parameters Class has all the required parameters for WikiImporter 7. Parser parses and WikiImporterListener events are fired. The next step would be to create xwiki pages on the fly( one at a time - at every WikiImporterListener#endWikiPage() event. ) Remaining : API to import the generated WikiPages into XWiki Is it good idea to use a component which makes use of document bridge to import the Wiki Pages, Attachments etc.. into XWiki. If not, what will be the best way to do it? -- Arun Reddy
On Wed, Jan 6, 2010 at 20:22, Arun Reddy <vipunreddy.n@gmail.com> wrote:
Hi Devs,
I have implemented an API for Wiki Importer.
The main component is org.xwiki.wikiimporter.WikiImporter **WikiImporterDescriptor getDescriptor() **void importWiki(Object object, WikiImporterListener listener) **WikiImporterType getType()
Wiki Call back event listener: org.xwiki.wikiimporter.WikiImporterListener **void beginWikiPage(String pageName, Map<String, String> params)
How Map<String, String> params is used ? If it's supposed to receive document informations like author, date, etc... i think it's should be well defined and typed parameters in a WikiPageParameters (otherwise there would be too much parameters in the method and it would not be easy to extends) instead of Map<String, String> params for example. Otherwise i don't see how you can use them if each importer can send any kind of document metadata. You can keep Map<String, String> params as custom parameters but we need at least a bunch of known page metadatas in a WikiPageParameters.
**void beginWikiPageRevision(String pageName, int revision, Map<String, String> params) **void beginObject(String objectType, Map<String, String> params) **void onProperty(String property, Map<String, String> params, String value) **void endObject(String objectType, Map<String, String> params) **void endWikiPageRevision(String pageName, int revision, Map<String, String> params) **void endWikiPage(String pageName, Map<String, String> params) **void beginAttachment(String attachmentName, Map<String, String> param) **void onAttachmentRevision(String attachmentName, Map<String, String> params, InputSource input) **void endAttachment(String attachmentName, Map<String, String> params)
Something is not explicit in your mail: WikiImporterListener extends org.xwiki.rendering.listener.Listener to send page content events right ?
WikiImporter parser for parsing documents (eg: XML ) : org.xwiki.wikiimporter.WikiImporterParser **void parse(InputSource source, WikiImporterListener listener) throws WikiImporterParseException
This sound like an implementation details. From public API user POV WikiImporter#importWiki(Parameters Class Object, WikiImporterListener ) is enough no need to go deeper, the importer then parse documents internally the way it wants to. Another reason is that "InputSource source" would be way too limited: for example I'm thinking of importers getting resources from another wiki directly online (XMLRPC, REST, etc...), it could also be a folder containing several files for each document (like content in a txt and objects in xml files), etc.
WikiImporter velocity bridge : org.xwiki.wikiimporter.WikiImporterVelocityBridge **public WikiImporterTypeFactory getWikiImporterTypeFactory() **public WikiImporter getWikiImporter(String wikiImporterType) throws WikiImporterException
WikiImporter type factory has methods to handle the supported other wiki formats/types. : org.xwiki.wikiimporter.WikiImporterTypeFactory **WikiImporterType createTypeFromIdString(String wikiImporterType) throws WikiImporterException **List<WikiImporterType> getAvailableTypes() throws WikiImporterException
This is a typical use case in my mind.
1. Get WikiImporterFactory instance with WikiImporterVelocityBridge#getWikiImporterFactory 2. Call the WikiImporterTypeFactory#getAvailableTypes() --> can be used at UI level to show the supported types ( MediaWiki XML, XMLRPC etc .. ) 3. Select the format type and call WikiImporterVelocityBridge#getWikiImporter(String wikiImporterType) to get selected WikiImporter instance. 4. Use the WikiImporterDescriptor to generate the UI. ( Similiar to MacroDescriptor) 5. User enters all the parameters ( UI ) 6. Call WikiImporter#importWiki(Parameters Class Object, WikiImporterListener ) to start the import process. Parameters Class has all the required parameters for WikiImporter 7. Parser parses and WikiImporterListener events are fired.
The next step would be to create xwiki pages on the fly( one at a time - at every WikiImporterListener#endWikiPage() event. )
Remaining : API to import the generated WikiPages into XWiki
Is it good idea to use a component which makes use of document bridge to import the Wiki Pages, Attachments etc.. into XWiki. If not, what will be the best way to do it?
Yes you should have a component for that but implemented in a separate module that depends on xwiki-core because the bridge is too limited for that so and it will be a pain. We can later refactor the implementation to use the new model and it's ready.
-- Arun Reddy _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne
Hi, On Fri, Jan 8, 2010 at 4:58 PM, Thomas Mortagne <thomas.mortagne@xwiki.com>wrote:
On Wed, Jan 6, 2010 at 20:22, Arun Reddy <vipunreddy.n@gmail.com> wrote:
Hi Devs,
I have implemented an API for Wiki Importer.
The main component is org.xwiki.wikiimporter.WikiImporter **WikiImporterDescriptor getDescriptor() **void importWiki(Object object, WikiImporterListener listener) **WikiImporterType getType()
Wiki Call back event listener: org.xwiki.wikiimporter.WikiImporterListener **void beginWikiPage(String pageName, Map<String, String> params)
How Map<String, String> params is used ? If it's supposed to receive document informations like author, date, etc... i think it's should be well defined and typed parameters in a WikiPageParameters (otherwise there would be too much parameters in the method and it would not be easy to extends) instead of Map<String, String> params for example. Otherwise i don't see how you can use them if each importer can send any kind of document metadata. You can keep Map<String, String> params as custom parameters but we need at least a bunch of known page metadatas in a WikiPageParameters.
No, not for author or date etc.. Some wiki specific metadata related to page and other elements, like Id, ClassName. ( in short element attributes ). As org.xwiki.rendering.Listener follows Map<String,String> parameters. I have followed the same.
In almost all the cases, the metadata ( author,date, comments, labels/tags etc.. ) is handled with onProperty() event. But its a good idea to use WikiPageParameters. I shall implement it.
**void beginWikiPageRevision(String pageName, int revision, Map<String, String> params) **void beginObject(String objectType, Map<String, String> params) **void onProperty(String property, Map<String, String> params, String value) **void endObject(String objectType, Map<String, String> params) **void endWikiPageRevision(String pageName, int revision, Map<String, String> params) **void endWikiPage(String pageName, Map<String, String> params) **void beginAttachment(String attachmentName, Map<String, String> param) **void onAttachmentRevision(String attachmentName, Map<String, String> params, InputSource input) **void endAttachment(String attachmentName, Map<String, String> params)
Something is not explicit in your mail: WikiImporterListener extends org.xwiki.rendering.listener.Listener to send page content events right ?
The WikiImporterListener doesn't extend org.xwiki.rendering.listener.Listener , because i never felt the need for all those methods. I have come up with a custom listener in the lines of org.xwiki.rendering.listener.Listener
WikiImporter parser for parsing documents (eg: XML ) : org.xwiki.wikiimporter.WikiImporterParser **void parse(InputSource source, WikiImporterListener listener) throws WikiImporterParseException
This sound like an implementation details. From public API user POV WikiImporter#importWiki(Parameters Class Object, WikiImporterListener ) is enough no need to go deeper, the importer then parse documents internally the way it wants to. Another reason is that "InputSource source" would be way too limited: for example I'm thinking of importers getting resources from another wiki directly online (XMLRPC, REST, etc...), it could also be a folder containing several files for each document (like content in a txt and objects in xml files), etc.
Ok, Yes InputSource can't handle all the cases.
Can i use a custom WikiImporterInputSource extending InputSource class.?
WikiImporter velocity bridge : org.xwiki.wikiimporter.WikiImporterVelocityBridge **public WikiImporterTypeFactory getWikiImporterTypeFactory() **public WikiImporter getWikiImporter(String wikiImporterType) throws WikiImporterException
WikiImporter type factory has methods to handle the supported other wiki formats/types. : org.xwiki.wikiimporter.WikiImporterTypeFactory **WikiImporterType createTypeFromIdString(String wikiImporterType) throws WikiImporterException **List<WikiImporterType> getAvailableTypes() throws WikiImporterException
This is a typical use case in my mind.
1. Get WikiImporterFactory instance with WikiImporterVelocityBridge#getWikiImporterFactory 2. Call the WikiImporterTypeFactory#getAvailableTypes() --> can be used at UI level to show the supported types ( MediaWiki XML, XMLRPC etc .. ) 3. Select the format type and call WikiImporterVelocityBridge#getWikiImporter(String wikiImporterType) to get selected WikiImporter instance. 4. Use the WikiImporterDescriptor to generate the UI. ( Similiar to MacroDescriptor) 5. User enters all the parameters ( UI ) 6. Call WikiImporter#importWiki(Parameters Class Object, WikiImporterListener ) to start the import process. Parameters Class has all the required parameters for WikiImporter 7. Parser parses and WikiImporterListener events are fired.
The next step would be to create xwiki pages on the fly( one at a time - at every WikiImporterListener#endWikiPage() event. )
Remaining : API to import the generated WikiPages into XWiki
Is it good idea to use a component which makes use of document bridge to import the Wiki Pages, Attachments etc.. into XWiki. If not, what will be the best way to do it?
Yes you should have a component for that but implemented in a separate module that depends on xwiki-core because the bridge is too limited for that so and it will be a pain. We can later refactor the implementation to use the new model and it's ready.
-- Arun Reddy _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Arun Reddy
On Sat, Jan 9, 2010 at 06:11, Arun Reddy <vipunreddy.n@gmail.com> wrote:
Hi,
On Fri, Jan 8, 2010 at 4:58 PM, Thomas Mortagne <thomas.mortagne@xwiki.com>wrote:
On Wed, Jan 6, 2010 at 20:22, Arun Reddy <vipunreddy.n@gmail.com> wrote:
Hi Devs,
I have implemented an API for Wiki Importer.
The main component is org.xwiki.wikiimporter.WikiImporter **WikiImporterDescriptor getDescriptor() **void importWiki(Object object, WikiImporterListener listener) **WikiImporterType getType()
Wiki Call back event listener: org.xwiki.wikiimporter.WikiImporterListener **void beginWikiPage(String pageName, Map<String, String> params)
How Map<String, String> params is used ? If it's supposed to receive document informations like author, date, etc... i think it's should be well defined and typed parameters in a WikiPageParameters (otherwise there would be too much parameters in the method and it would not be easy to extends) instead of Map<String, String> params for example. Otherwise i don't see how you can use them if each importer can send any kind of document metadata. You can keep Map<String, String> params as custom parameters but we need at least a bunch of known page metadatas in a WikiPageParameters.
No, not for author or date etc.. Some wiki specific metadata related to page and other elements, like Id, ClassName. ( in short element attributes ). As org.xwiki.rendering.Listener follows Map<String,String> parameters. I have followed the same.
In almost all the cases, the metadata ( author,date, comments, labels/tags etc.. ) is handled with onProperty() event.
But its a good idea to use WikiPageParameters. I shall implement it.
> **void beginWikiPageRevision(String pageName, int revision, Map<String,
String> params) **void beginObject(String objectType, Map<String, String> params) **void onProperty(String property, Map<String, String> params, String value) **void endObject(String objectType, Map<String, String> params) **void endWikiPageRevision(String pageName, int revision, Map<String, String> params) **void endWikiPage(String pageName, Map<String, String> params) **void beginAttachment(String attachmentName, Map<String, String> param) **void onAttachmentRevision(String attachmentName, Map<String, String> params, InputSource input) **void endAttachment(String attachmentName, Map<String, String> params)
Something is not explicit in your mail: WikiImporterListener extends org.xwiki.rendering.listener.Listener to send page content events right ?
The WikiImporterListener doesn't extend org.xwiki.rendering.listener.Listener , because i never felt the need for all those methods. I have come up with a custom listener in the lines of org.xwiki.rendering.listener.Listener
The point of extending Listener is to handle document content. For example a client would receive something like: beginWikiPage beginWikiPageRevision beginDocument beginParagraph onWord onSpace onWord endParagraph endDocument endWikiPageRevision endPage beginDocument -> endDocument are document content events in a generic format (the event version of XDOM). That way the client receive everything he needs to reconstruct the whole wiki in a purely generic form (no specific syntax), it's WikiImporter responsibility to do all the needed parsing and conversion to send the proper events to send valid events representing a complete wiki in WikiImporterListener. Said another way: Listener describe a document content and WikiImporterListener extends it to describe the whole wiki. It's not clear in your proposal how you where planning to send the document content. Is it one of the onProperty ?
WikiImporter parser for parsing documents (eg: XML ) : org.xwiki.wikiimporter.WikiImporterParser **void parse(InputSource source, WikiImporterListener listener) throws WikiImporterParseException
This sound like an implementation details. From public API user POV WikiImporter#importWiki(Parameters Class Object, WikiImporterListener ) is enough no need to go deeper, the importer then parse documents internally the way it wants to. Another reason is that "InputSource source" would be way too limited: for example I'm thinking of importers getting resources from another wiki directly online (XMLRPC, REST, etc...), it could also be a folder containing several files for each document (like content in a txt and objects in xml files), etc.
Ok, Yes InputSource can't handle all the cases.
Can i use a custom WikiImporterInputSource extending InputSource class.?
No, what i mean is that WikiImporterParser#parse is useless in the API. The user of the wiki importer should only call WikiImporter#importWiki(Parameters Class Object, WikiImporterListener) and receive all needed events for the whole wiki. I don't see the point of having this WikiImporterParser#parse method at all.
>
WikiImporter velocity bridge : org.xwiki.wikiimporter.WikiImporterVelocityBridge **public WikiImporterTypeFactory getWikiImporterTypeFactory() **public WikiImporter getWikiImporter(String wikiImporterType) throws WikiImporterException
WikiImporter type factory has methods to handle the supported other wiki formats/types. : org.xwiki.wikiimporter.WikiImporterTypeFactory **WikiImporterType createTypeFromIdString(String wikiImporterType) throws WikiImporterException **List<WikiImporterType> getAvailableTypes() throws WikiImporterException
This is a typical use case in my mind.
1. Get WikiImporterFactory instance with WikiImporterVelocityBridge#getWikiImporterFactory 2. Call the WikiImporterTypeFactory#getAvailableTypes() --> can be used at UI level to show the supported types ( MediaWiki XML, XMLRPC etc .. ) 3. Select the format type and call WikiImporterVelocityBridge#getWikiImporter(String wikiImporterType) to get selected WikiImporter instance. 4. Use the WikiImporterDescriptor to generate the UI. ( Similiar to MacroDescriptor) 5. User enters all the parameters ( UI ) 6. Call WikiImporter#importWiki(Parameters Class Object, WikiImporterListener ) to start the import process. Parameters Class has all the required parameters for WikiImporter 7. Parser parses and WikiImporterListener events are fired.
The next step would be to create xwiki pages on the fly( one at a time - at every WikiImporterListener#endWikiPage() event. )
Remaining : API to import the generated WikiPages into XWiki
Is it good idea to use a component which makes use of document bridge to import the Wiki Pages, Attachments etc.. into XWiki. If not, what will be the best way to do it?
Yes you should have a component for that but implemented in a separate module that depends on xwiki-core because the bridge is too limited for that so and it will be a pain. We can later refactor the implementation to use the new model and it's ready.
-- Arun Reddy _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Arun Reddy _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne
Hi, On Sun, Jan 10, 2010 at 4:04 AM, Thomas Mortagne <thomas.mortagne@xwiki.com>wrote:
On Sat, Jan 9, 2010 at 06:11, Arun Reddy <vipunreddy.n@gmail.com> wrote:
Hi,
On Fri, Jan 8, 2010 at 4:58 PM, Thomas Mortagne <thomas.mortagne@xwiki.com>wrote:
On Wed, Jan 6, 2010 at 20:22, Arun Reddy <vipunreddy.n@gmail.com> wrote:
Hi Devs,
I have implemented an API for Wiki Importer.
The main component is org.xwiki.wikiimporter.WikiImporter **WikiImporterDescriptor getDescriptor() **void importWiki(Object object, WikiImporterListener listener) **WikiImporterType getType()
Wiki Call back event listener: org.xwiki.wikiimporter.WikiImporterListener **void beginWikiPage(String pageName, Map<String, String> params)
How Map<String, String> params is used ? If it's supposed to receive document informations like author, date, etc... i think it's should be well defined and typed parameters in a WikiPageParameters (otherwise there would be too much parameters in the method and it would not be easy to extends) instead of Map<String, String> params for example. Otherwise i don't see how you can use them if each importer can send any kind of document metadata. You can keep Map<String, String> params as custom parameters but we need at least a bunch of known page metadatas in a WikiPageParameters.
No, not for author or date etc.. Some wiki specific metadata related to page and other elements, like Id, ClassName. ( in short element attributes ). As org.xwiki.rendering.Listener follows Map<String,String> parameters. I have followed the same.
In almost all the cases, the metadata ( author,date, comments, labels/tags etc.. ) is handled with onProperty() event.
But its a good idea to use WikiPageParameters. I shall implement it.
**void beginWikiPageRevision(String pageName, int revision, Map<String, String> params) **void beginObject(String objectType, Map<String, String> params) **void onProperty(String property, Map<String, String> params, String value) **void endObject(String objectType, Map<String, String> params) **void endWikiPageRevision(String pageName, int revision, Map<String, String> params) **void endWikiPage(String pageName, Map<String, String> params) **void beginAttachment(String attachmentName, Map<String, String> param) **void onAttachmentRevision(String attachmentName, Map<String, String> params, InputSource input) **void endAttachment(String attachmentName, Map<String, String> params)
Something is not explicit in your mail: WikiImporterListener extends org.xwiki.rendering.listener.Listener to send page content events right ?
The WikiImporterListener doesn't extend org.xwiki.rendering.listener.Listener , because i never felt the need for all those methods. I have come up with a custom listener in the lines of org.xwiki.rendering.listener.Listener
The point of extending Listener is to handle document content. For example a client would receive something like:
beginWikiPage beginWikiPageRevision beginDocument beginParagraph onWord onSpace onWord endParagraph endDocument endWikiPageRevision endPage
beginDocument -> endDocument are document content events in a generic format (the event version of XDOM). That way the client receive everything he needs to reconstruct the whole wiki in a purely generic form (no specific syntax), it's WikiImporter responsibility to do all the needed parsing and conversion to send the proper events to send valid events representing a complete wiki in WikiImporterListener.
Said another way: Listener describe a document content and WikiImporterListener extends it to describe the whole wiki.
It's not clear in your proposal how you where planning to send the document content. Is it one of the onProperty ?
Yes, all the document content is handled using onProperty Tag. But extending Listener and implementing XDOM events makes it more complete ( Its missing in the proposed API ) :)
WikiImporter parser for parsing documents (eg: XML ) : org.xwiki.wikiimporter.WikiImporterParser **void parse(InputSource source, WikiImporterListener listener) throws WikiImporterParseException
This sound like an implementation details. From public API user POV WikiImporter#importWiki(Parameters Class Object, WikiImporterListener ) is enough no need to go deeper, the importer then parse documents internally the way it wants to. Another reason is that "InputSource source" would be way too limited: for example I'm thinking of importers getting resources from another wiki directly online (XMLRPC, REST, etc...), it could also be a folder containing several files for each document (like content in a txt and objects in xml files), etc.
Ok, Yes InputSource can't handle all the cases.
Can i use a custom WikiImporterInputSource extending InputSource class.?
No, what i mean is that WikiImporterParser#parse is useless in the API. The user of the wiki importer should only call WikiImporter#importWiki(Parameters Class Object, WikiImporterListener) and receive all needed events for the whole wiki. I don't see the point of having this WikiImporterParser#parse method at all.
Said that its useless to have WikiImporterParser#parse method. It shouldnt be exposed to WikiImporter User in API (or) not to have it internally, i mean WikiImporter#importWiki(Parameters Class Object, WikiImporterListener) makes call to WikiImporterParser#parse internally inside its implementation. ?
I have the second one in my mind.
WikiImporter velocity bridge : org.xwiki.wikiimporter.WikiImporterVelocityBridge **public WikiImporterTypeFactory getWikiImporterTypeFactory() **public WikiImporter getWikiImporter(String wikiImporterType) throws WikiImporterException
WikiImporter type factory has methods to handle the supported other
wiki
formats/types. : org.xwiki.wikiimporter.WikiImporterTypeFactory **WikiImporterType createTypeFromIdString(String wikiImporterType) throws WikiImporterException **List<WikiImporterType> getAvailableTypes() throws WikiImporterException
This is a typical use case in my mind.
1. Get WikiImporterFactory instance with WikiImporterVelocityBridge#getWikiImporterFactory 2. Call the WikiImporterTypeFactory#getAvailableTypes() --> can be used at UI level to show the supported types ( MediaWiki XML, XMLRPC etc .. ) 3. Select the format type and call WikiImporterVelocityBridge#getWikiImporter(String wikiImporterType) to get selected WikiImporter instance. 4. Use the WikiImporterDescriptor to generate the UI. ( Similiar to MacroDescriptor) 5. User enters all the parameters ( UI ) 6. Call WikiImporter#importWiki(Parameters Class Object, WikiImporterListener ) to start the import process. Parameters Class has all the required parameters for WikiImporter 7. Parser parses and WikiImporterListener events are fired.
The next step would be to create xwiki pages on the fly( one at a time - at every WikiImporterListener#endWikiPage() event. )
Remaining : API to import the generated WikiPages into XWiki
Is it good idea to use a component which makes use of document bridge to import the Wiki Pages, Attachments etc.. into XWiki. If not, what will be the best way to do it?
Yes you should have a component for that but implemented in a separate module that depends on xwiki-core because the bridge is too limited for that so and it will be a pain. We can later refactor the implementation to use the new model and it's ready.
-- Arun Reddy _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Arun Reddy _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Arun Reddy
On Sun, Jan 10, 2010 at 05:57, Arun Reddy <vipunreddy.n@gmail.com> wrote:
Hi,
On Sun, Jan 10, 2010 at 4:04 AM, Thomas Mortagne <thomas.mortagne@xwiki.com>wrote:
On Sat, Jan 9, 2010 at 06:11, Arun Reddy <vipunreddy.n@gmail.com> wrote:
Hi,
On Fri, Jan 8, 2010 at 4:58 PM, Thomas Mortagne <thomas.mortagne@xwiki.com>wrote:
On Wed, Jan 6, 2010 at 20:22, Arun Reddy <vipunreddy.n@gmail.com> wrote:
Hi Devs,
I have implemented an API for Wiki Importer.
The main component is org.xwiki.wikiimporter.WikiImporter **WikiImporterDescriptor getDescriptor() **void importWiki(Object object, WikiImporterListener listener) **WikiImporterType getType()
Wiki Call back event listener: org.xwiki.wikiimporter.WikiImporterListener **void beginWikiPage(String pageName, Map<String, String> params)
How Map<String, String> params is used ? If it's supposed to receive document informations like author, date, etc... i think it's should be well defined and typed parameters in a WikiPageParameters (otherwise there would be too much parameters in the method and it would not be easy to extends) instead of Map<String, String> params for example. Otherwise i don't see how you can use them if each importer can send any kind of document metadata. You can keep Map<String, String> params as custom parameters but we need at least a bunch of known page metadatas in a WikiPageParameters.
No, not for author or date etc.. Some wiki specific metadata related to page and other elements, like Id, ClassName. ( in short element attributes ). As org.xwiki.rendering.Listener follows Map<String,String> parameters. I have followed the same.
In almost all the cases, the metadata ( author,date, comments, labels/tags etc.. ) is handled with onProperty() event.
But its a good idea to use WikiPageParameters. I shall implement it.
> **void beginWikiPageRevision(String pageName, int revision, Map<String,
String> params) **void beginObject(String objectType, Map<String, String> params) **void onProperty(String property, Map<String, String> params, String value) **void endObject(String objectType, Map<String, String> params) **void endWikiPageRevision(String pageName, int revision, Map<String, String> params) **void endWikiPage(String pageName, Map<String, String> params) **void beginAttachment(String attachmentName, Map<String, String> param) **void onAttachmentRevision(String attachmentName, Map<String, String> params, InputSource input) **void endAttachment(String attachmentName, Map<String, String> params)
Something is not explicit in your mail: WikiImporterListener extends org.xwiki.rendering.listener.Listener to send page content events right ?
The WikiImporterListener doesn't extend org.xwiki.rendering.listener.Listener , because i never felt the need for all those methods. I have come up with a custom listener in the lines of org.xwiki.rendering.listener.Listener
The point of extending Listener is to handle document content. For example a client would receive something like:
beginWikiPage beginWikiPageRevision beginDocument beginParagraph onWord onSpace onWord endParagraph endDocument endWikiPageRevision endPage
beginDocument -> endDocument are document content events in a generic format (the event version of XDOM). That way the client receive everything he needs to reconstruct the whole wiki in a purely generic form (no specific syntax), it's WikiImporter responsibility to do all the needed parsing and conversion to send the proper events to send valid events representing a complete wiki in WikiImporterListener.
Said another way: Listener describe a document content and WikiImporterListener extends it to describe the whole wiki.
It's not clear in your proposal how you where planning to send the document content. Is it one of the onProperty ?
Yes, all the document content is handled using onProperty Tag. But extending Listener and implementing XDOM events makes it more complete ( Its missing in the proposed API ) :)
WikiImporter parser for parsing documents (eg: XML ) : org.xwiki.wikiimporter.WikiImporterParser **void parse(InputSource source, WikiImporterListener listener) throws WikiImporterParseException
This sound like an implementation details. From public API user POV WikiImporter#importWiki(Parameters Class Object, WikiImporterListener ) is enough no need to go deeper, the importer then parse documents internally the way it wants to. Another reason is that "InputSource source" would be way too limited: for example I'm thinking of importers getting resources from another wiki directly online (XMLRPC, REST, etc...), it could also be a folder containing several files for each document (like content in a txt and objects in xml files), etc.
Ok, Yes InputSource can't handle all the cases.
Can i use a custom WikiImporterInputSource extending InputSource class.?
No, what i mean is that WikiImporterParser#parse is useless in the API. The user of the wiki importer should only call WikiImporter#importWiki(Parameters Class Object, WikiImporterListener) and receive all needed events for the whole wiki. I don't see the point of having this WikiImporterParser#parse method at all.
Said that its useless to have WikiImporterParser#parse method. It shouldnt be exposed to WikiImporter User in API (or) not to have it internally, i mean WikiImporter#importWiki(Parameters Class Object, WikiImporterListener) makes call to WikiImporterParser#parse internally inside its implementation.
A wiki importer implementation is doing anything it wants internally, my point is that it should not be part of the public API.
?
I have the second one in my mind.
>
>
WikiImporter velocity bridge : org.xwiki.wikiimporter.WikiImporterVelocityBridge **public WikiImporterTypeFactory getWikiImporterTypeFactory() **public WikiImporter getWikiImporter(String wikiImporterType) throws WikiImporterException
WikiImporter type factory has methods to handle the supported other wiki formats/types. : org.xwiki.wikiimporter.WikiImporterTypeFactory **WikiImporterType createTypeFromIdString(String wikiImporterType) throws WikiImporterException **List<WikiImporterType> getAvailableTypes() throws WikiImporterException
This is a typical use case in my mind.
1. Get WikiImporterFactory instance with WikiImporterVelocityBridge#getWikiImporterFactory 2. Call the WikiImporterTypeFactory#getAvailableTypes() --> can be used at UI level to show the supported types ( MediaWiki XML, XMLRPC etc .. ) 3. Select the format type and call WikiImporterVelocityBridge#getWikiImporter(String wikiImporterType) to get selected WikiImporter instance. 4. Use the WikiImporterDescriptor to generate the UI. ( Similiar to MacroDescriptor) 5. User enters all the parameters ( UI ) 6. Call WikiImporter#importWiki(Parameters Class Object, WikiImporterListener ) to start the import process. Parameters Class has all the required parameters for WikiImporter 7. Parser parses and WikiImporterListener events are fired.
The next step would be to create xwiki pages on the fly( one at a time - at every WikiImporterListener#endWikiPage() event. )
Remaining : API to import the generated WikiPages into XWiki
Is it good idea to use a component which makes use of document bridge to import the Wiki Pages, Attachments etc.. into XWiki. If not, what will be the best way to do it?
Yes you should have a component for that but implemented in a separate module that depends on xwiki-core because the bridge is too limited for that so and it will be a pain. We can later refactor the implementation to use the new model and it's ready.
-- Arun Reddy _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Arun Reddy _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Arun Reddy _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne
participants (3)
-
Arun Reddy -
Asiri Rathnayake -
Thomas Mortagne