[xwiki-devs] XAR source projects should allow source files
Hello fellow developers, many of you are developing XAR-packaged resources. The exchangeability of these archives is one of the precious aspects of XWiki, I believe. However, if one see all the many GitHub-contributed xar packages, the source is not there: it's in a Wiki with which one would "open it". A pity since much of the content is code. I'd like to propose to allow XARs to be built form source code which is true source that one edits. Code files with their proper syntax coloring, editors, auto-completion, search. Media-files that are edited with our tool of choice by a simple double-click and not a complex pull, edit, and replace. Thus, my proposal is to allow inclusions within the XML files. Inclusions should be of type: - text to include page content or object properties (e.g. css of cssx objects) - attachments to include attached objects My current setup is documented here:http://jira.xwiki.org/browse/XWIKI-13643 It does binaryInclude to generate base64 and textInclude to generate XML-text. Thus, I can advertise a github project which is a macro and whose complex function is in a velocity file that is in a .vm file. However, I do not need to explode everything, e.g. the text-content describing the macro or (in this case too small) translation set. What do you think? Should I rather generate complete attachment elements (name, length, modif date, ...) instead of doing a binaryInclude that just generates base64 ? Is fetching files for inclusion beyond the boundaries of the project of any use? (it is currently prohibited for safety) thanks in advance. Paul
Not that an alternative is to add a mojo in the XAR plugin (or write a new Maven plugin) based on Filter module to produce a XAR package from http://extensions.xwiki.org/xwiki/bin/view/Extension/XFF+filter which already been designed for what you want: provide a format that can be edited from filesystem easily. In theory all it takes is to put XFF filter in input and XAR filter in output and run. On Mon, Aug 22, 2016 at 3:59 PM, Paul Libbrecht <paul@hoplahup.net> wrote:
Hello fellow developers,
many of you are developing XAR-packaged resources. The exchangeability of these archives is one of the precious aspects of XWiki, I believe.
However, if one see all the many GitHub-contributed xar packages, the source is not there: it's in a Wiki with which one would "open it". A pity since much of the content is code.
I'd like to propose to allow XARs to be built form source code which is true source that one edits. Code files with their proper syntax coloring, editors, auto-completion, search. Media-files that are edited with our tool of choice by a simple double-click and not a complex pull, edit, and replace.
Thus, my proposal is to allow inclusions within the XML files. Inclusions should be of type: - text to include page content or object properties (e.g. css of cssx objects) - attachments to include attached objects
My current setup is documented here:http://jira.xwiki.org/browse/XWIKI-13643 It does binaryInclude to generate base64 and textInclude to generate XML-text. Thus, I can advertise a github project which is a macro and whose complex function is in a velocity file that is in a .vm file. However, I do not need to explode everything, e.g. the text-content describing the macro or (in this case too small) translation set.
What do you think? Should I rather generate complete attachment elements (name, length, modif date, ...) instead of doing a binaryInclude that just generates base64 ? Is fetching files for inclusion beyond the boundaries of the project of any use? (it is currently prohibited for safety)
thanks in advance.
Paul _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne
Thomas, I had seen this approach yesterday only. I think my attempt is a tick more conservative. But indeed, it seems to be the objective of the XFF filter. Also, what I could not tell fully from the XFF attempt is whether the optionality was there: are there property values or even complete objects that I can leave gouped in some page XML? It seems to me that itreally does not make sense to include them all in a file each. (e.g. objects of the XWiki.TranslationDocumentClass). paul Thomas Mortagne wrote:
Note that an alternative is to add a mojo in the XAR plugin (or write a new Maven plugin) based on Filter module to produce a XAR package from http://extensions.xwiki.org/xwiki/bin/view/Extension/XFF+filter which already been designed for what you want: provide a format that can be edited from filesystem easily. In theory all it takes is to put XFF filter in input and XAR filter in output and run.
I can't give you much details on the XFF format, I just know the general goal. I only helped Jean on the "how to use Filter API" side. On Tue, Aug 23, 2016 at 10:57 AM, Paul Libbrecht <paul@hoplahup.net> wrote:
Thomas,
I had seen this approach yesterday only. I think my attempt is a tick more conservative. But indeed, it seems to be the objective of the XFF filter.
Also, what I could not tell fully from the XFF attempt is whether the optionality was there: are there property values or even complete objects that I can leave gouped in some page XML? It seems to me that itreally does not make sense to include them all in a file each. (e.g. objects of the XWiki.TranslationDocumentClass).
paul
Thomas Mortagne wrote:
Note that an alternative is to add a mojo in the XAR plugin (or write a new Maven plugin) based on Filter module to produce a XAR package from http://extensions.xwiki.org/xwiki/bin/view/Extension/XFF+filter which already been designed for what you want: provide a format that can be edited from filesystem easily. In theory all it takes is to put XFF filter in input and XAR filter in output and run.
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne
The format seems to be defined here: http://extensions.xwiki.org/xwiki/bin/view/Extension/XFF+filter#HFormat Thanks -Vincent
On 23 Aug 2016, at 11:00, Thomas Mortagne <thomas.mortagne@xwiki.com> wrote:
I can't give you much details on the XFF format, I just know the general goal. I only helped Jean on the "how to use Filter API" side.
On Tue, Aug 23, 2016 at 10:57 AM, Paul Libbrecht <paul@hoplahup.net> wrote:
Thomas,
I had seen this approach yesterday only. I think my attempt is a tick more conservative. But indeed, it seems to be the objective of the XFF filter.
Also, what I could not tell fully from the XFF attempt is whether the optionality was there: are there property values or even complete objects that I can leave gouped in some page XML? It seems to me that itreally does not make sense to include them all in a file each. (e.g. objects of the XWiki.TranslationDocumentClass).
paul
Thomas Mortagne wrote:
Note that an alternative is to add a mojo in the XAR plugin (or write a new Maven plugin) based on Filter module to produce a XAR package from http://extensions.xwiki.org/xwiki/bin/view/Extension/XFF+filter which already been designed for what you want: provide a format that can be edited from filesystem easily. In theory all it takes is to put XFF filter in input and XAR filter in output and run.
__________
Let's ask things differently: what's missing XFF for others to use? paul
Vincent Massol <mailto:vincent@massol.net> 23 August 2016 at 11:01 The format seems to be defined here: http://extensions.xwiki.org/xwiki/bin/view/Extension/XFF+filter#HFormat
On 23 Aug 2016, at 11:13, Paul Libbrecht <paul@hoplahup.net> wrote:
Let's ask things differently: what's missing XFF for others to use?
I think it’s just missing a maven plugin to use it. All the other parts are ready. Thanks -Vincent
paul
Vincent Massol <mailto:vincent@massol.net> 23 August 2016 at 11:01 The format seems to be defined here: http://extensions.xwiki.org/xwiki/bin/view/Extension/XFF+filter#HFormat
On 23 Aug 2016, at 11:15, Vincent Massol <vincent@massol.net> wrote:
On 23 Aug 2016, at 11:13, Paul Libbrecht <paul@hoplahup.net> wrote:
Let's ask things differently: what's missing XFF for others to use?
I think it’s just missing a maven plugin to use it. All the other parts are ready.
Actually just noticed that Jean has started one: https://github.com/xwiki-contrib/api-xff/tree/master/xff-maven I don’t know its status though. So what needs to be done is try the full chain and see what works/what doesn’t. Unless jean could tell us what’s the status? (I’ll ping him on skype). Thanks -Vincent
Thanks -Vincent
paul
Vincent Massol <mailto:vincent@massol.net> 23 August 2016 at 11:01 The format seems to be defined here: http://extensions.xwiki.org/xwiki/bin/view/Extension/XFF+filter#HFormat
I've just tried it. http://direct.hoplahup.net/tmp/xffAttempt-first.zip After having checked-out https://github.com/xwiki-contrib/api-xff and run mvn -Denforcer.skip=true install. What I obtain after mvn package is a file my-application.xff. I can't seem to import that. Should I be able to? thanks Paul
Vincent Massol <mailto:vincent@massol.net> 23 August 2016 at 11:35
On 23 Aug 2016, at 11:15, Vincent Massol <vincent@massol.net> wrote:
On 23 Aug 2016, at 11:13, Paul Libbrecht <paul@hoplahup.net> wrote:
Let's ask things differently: what's missing XFF for others to use? I think it’s just missing a maven plugin to use it. All the other parts are ready.
Actually just noticed that Jean has started one: https://github.com/xwiki-contrib/api-xff/tree/master/xff-maven
I don’t know its status though.
So what needs to be done is try the full chain and see what works/what doesn’t.
Unless jean could tell us what’s the status? (I’ll ping him on skype).
Thanks -Vincent
Thanks -Vincent
paul
Vincent Massol <mailto:vincent@massol.net> 23 August 2016 at 11:01 The format seems to be defined here: http://extensions.xwiki.org/xwiki/bin/view/Extension/XFF+filter#HFormat
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs Vincent Massol <mailto:vincent@massol.net> 23 August 2016 at 11:15
On 23 Aug 2016, at 11:13, Paul Libbrecht <paul@hoplahup.net> wrote:
Let's ask things differently: what's missing XFF for others to use?
I think it’s just missing a maven plugin to use it. All the other parts are ready.
Thanks -Vincent
paul
Vincent Massol <mailto:vincent@massol.net> 23 August 2016 at 11:01 The format seems to be defined here: http://extensions.xwiki.org/xwiki/bin/view/Extension/XFF+filter#HFormat
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs Paul Libbrecht <mailto:paul@hoplahup.net> 23 August 2016 at 11:13 Let's ask things differently: what's missing XFF for others to use?
paul
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs Vincent Massol <mailto:vincent@massol.net> 23 August 2016 at 11:01 The format seems to be defined here: http://extensions.xwiki.org/xwiki/bin/view/Extension/XFF+filter#HFormat
Thanks -Vincent
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs Thomas Mortagne <mailto:thomas.mortagne@xwiki.com> 23 August 2016 at 11:00 I can't give you much details on the XFF format, I just know the general goal. I only helped Jean on the "how to use Filter API" side.
On Tue, Aug 23, 2016 at 11:46 AM, Paul Libbrecht <paul@hoplahup.net> wrote:
I've just tried it. http://direct.hoplahup.net/tmp/xffAttempt-first.zip After having checked-out https://github.com/xwiki-contrib/api-xff and run mvn -Denforcer.skip=true install. What I obtain after mvn package is a file my-application.xff. I can't seem to import that. Should I be able to?
Not trough XAR tools no, it's completely different format.
thanks Paul
Vincent Massol <mailto:vincent@massol.net> 23 August 2016 at 11:35
On 23 Aug 2016, at 11:15, Vincent Massol <vincent@massol.net> wrote:
On 23 Aug 2016, at 11:13, Paul Libbrecht <paul@hoplahup.net> wrote:
Let's ask things differently: what's missing XFF for others to use? I think it’s just missing a maven plugin to use it. All the other parts are ready.
Actually just noticed that Jean has started one: https://github.com/xwiki-contrib/api-xff/tree/master/xff-maven
I don’t know its status though.
So what needs to be done is try the full chain and see what works/what doesn’t.
Unless jean could tell us what’s the status? (I’ll ping him on skype).
Thanks -Vincent
Thanks -Vincent
paul
Vincent Massol <mailto:vincent@massol.net> 23 August 2016 at 11:01 The format seems to be defined here: http://extensions.xwiki.org/xwiki/bin/view/Extension/XFF+filter#HFormat
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs Vincent Massol <mailto:vincent@massol.net> 23 August 2016 at 11:15
On 23 Aug 2016, at 11:13, Paul Libbrecht <paul@hoplahup.net> wrote:
Let's ask things differently: what's missing XFF for others to use?
I think it’s just missing a maven plugin to use it. All the other parts are ready.
Thanks -Vincent
paul
Vincent Massol <mailto:vincent@massol.net> 23 August 2016 at 11:01 The format seems to be defined here: http://extensions.xwiki.org/xwiki/bin/view/Extension/XFF+filter#HFormat
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs Paul Libbrecht <mailto:paul@hoplahup.net> 23 August 2016 at 11:13 Let's ask things differently: what's missing XFF for others to use?
paul
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs Vincent Massol <mailto:vincent@massol.net> 23 August 2016 at 11:01 The format seems to be defined here: http://extensions.xwiki.org/xwiki/bin/view/Extension/XFF+filter#HFormat
Thanks -Vincent
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs Thomas Mortagne <mailto:thomas.mortagne@xwiki.com> 23 August 2016 at 11:00 I can't give you much details on the XFF format, I just know the general goal. I only helped Jean on the "how to use Filter API" side.
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne
What I obtain after mvn package is a file my-application.xff.
I can't seem to import that. Should I be able to? Not trough XAR tools no, it's completely different format. So what needs to be done here is a converter which is a lot bigger than a packager. We'll see what Jean says.
In the meantime, do I understand at least that an enrichment to the xar-mojo is a more conservative approach that might have a better chance of uptake? What does it take for me to get the patch accepted? thanks Paul
On Tue, Aug 23, 2016 at 12:17 PM, Paul Libbrecht <paul@hoplahup.net> wrote:
What I obtain after mvn package is a file my-application.xff.
I can't seem to import that. Should I be able to? Not trough XAR tools no, it's completely different format. So what needs to be done here is a converter which is a lot bigger than a packager.
Which is what I indicated: you just use XFF filter (which already exist) in input and XAR filter (which also already exist) as output.
We'll see what Jean says.
In the meantime, do I understand at least that an enrichment to the xar-mojo is a more conservative approach that might have a better chance of uptake? What does it take for me to get the patch accepted?
thanks
Paul _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne
On Tue, Aug 23, 2016 at 12:26 PM, Thomas Mortagne <thomas.mortagne@xwiki.com> wrote:
On Tue, Aug 23, 2016 at 12:17 PM, Paul Libbrecht <paul@hoplahup.net> wrote:
What I obtain after mvn package is a file my-application.xff.
I can't seem to import that. Should I be able to? Not trough XAR tools no, it's completely different format. So what needs to be done here is a converter which is a lot bigger than a packager.
Which is what I indicated: you just use XFF filter (which already exist) in input and XAR filter (which also already exist) as output.
We'll see what Jean says.
In the meantime, do I understand at least that an enrichment to the xar-mojo is a more conservative approach that might have a better chance of uptake?
What does it take for me to get the patch accepted?
It takes the following IMO: * it works * it does not break any retro compatibility (probably not enabled by default for example) Now Guillaume has a point. The issue you will quickly have as soon as you start using this is that you HAVE to modify your XAR trough filesystem because you can't edit it in XWiki and export it anymore. That is if you don't provide any tool to export this kind of XAR extension.
thanks
Paul _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne
-- Thomas Mortagne
Thomas Mortagne wrote:
It takes the following IMO: * it works * it does not break any retro compatibility both are claimed. There are unit tests. The only danger of breaking anything is if dom4j does not faithfully restore the XML source. I believe this can be ignored just as it has been ignored by the implementation of transformations. (probably not enabled by default for example) No, it is enabled by default. I could replace the dom4j output by a file copy if no relevant elements are found.
Note however, that it operates on elements that were never allowed before.
Now Guillaume has a point. The issue you will quickly have as soon as you start using this is that you HAVE to modify your XAR trough filesystem because you can't edit it in XWiki and export it anymore. That is if you don't provide any tool to export this kind of XAR extension. I believe this is best practice to insure that the wiki does not insert you surprise elements such as the author name or the edit comment. Is this practice not corresponding at all what others are doing?
Using an XML-diff might be answering Guillaume's point... That's not completely trivial. Paul
2016-08-23 12:36 GMT+02:00 Paul Libbrecht <paul@hoplahup.net>:
Thomas Mortagne wrote:
It takes the following IMO: * it works * it does not break any retro compatibility both are claimed. There are unit tests. The only danger of breaking anything is if dom4j does not faithfully restore the XML source. I believe this can be ignored just as it has been ignored by the implementation of transformations. (probably not enabled by default for example) No, it is enabled by default. I could replace the dom4j output by a file copy if no relevant elements are found.
Note however, that it operates on elements that were never allowed before.
Now Guillaume has a point. The issue you will quickly have as soon as you start using this is that you HAVE to modify your XAR trough filesystem because you can't edit it in XWiki and export it anymore. That is if you don't provide any tool to export this kind of XAR extension. I believe this is best practice to insure that the wiki does not insert you surprise elements such as the author name or the edit comment.
In practice, this is the only manual step. I do a diff of the modifications I have made and I revert all changes that are meaningless. But it's quick & easy using my IDE (IntelliJ). I know some developers here do the same. The only issue is when the XAR format have changed: some fields are not ordered in the same way, and it makes harder to compute the real diff.
Is this practice not corresponding at all what others are doing?
Using an XML-diff might be answering Guillaume's point... That's not completely trivial.
Paul _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Guillaume Delhumeau (guillaume.delhumeau@xwiki.com) Research & Development Engineer at XWiki SAS Committer on the XWiki.org project
Of course, we could still change our workflow and copy/paste source contents manually. Not sure it's gonna be more effective. So I prefer the XFF approach (except that I would love having a Nested Document terminology instead of the current Space/Document one). 2016-08-23 13:11 GMT+02:00 Guillaume Delhumeau < guillaume.delhumeau@xwiki.com>:
2016-08-23 12:36 GMT+02:00 Paul Libbrecht <paul@hoplahup.net>:
Thomas Mortagne wrote:
It takes the following IMO: * it works * it does not break any retro compatibility both are claimed. There are unit tests. The only danger of breaking anything is if dom4j does not faithfully restore the XML source. I believe this can be ignored just as it has been ignored by the implementation of transformations. (probably not enabled by default for example) No, it is enabled by default. I could replace the dom4j output by a file copy if no relevant elements are found.
Note however, that it operates on elements that were never allowed before.
Now Guillaume has a point. The issue you will quickly have as soon as you start using this is that you HAVE to modify your XAR trough filesystem because you can't edit it in XWiki and export it anymore. That is if you don't provide any tool to export this kind of XAR extension. I believe this is best practice to insure that the wiki does not insert you surprise elements such as the author name or the edit comment.
In practice, this is the only manual step. I do a diff of the modifications I have made and I revert all changes that are meaningless. But it's quick & easy using my IDE (IntelliJ). I know some developers here do the same. The only issue is when the XAR format have changed: some fields are not ordered in the same way, and it makes harder to compute the real diff.
Is this practice not corresponding at all what others are doing?
Using an XML-diff might be answering Guillaume's point... That's not completely trivial.
Paul _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Guillaume Delhumeau (guillaume.delhumeau@xwiki.com) Research & Development Engineer at XWiki SAS Committer on the XWiki.org project
-- Guillaume Delhumeau (guillaume.delhumeau@xwiki.com) Research & Development Engineer at XWiki SAS Committer on the XWiki.org project
Guillaume, sure, this is *doable* but it can also be done at discretion. E.g. you'd only copy from the exploded xar the other files you copied but not the source. Also, if you replace the XML files with those of the XARs, you'll just ignore the sources, since you've removed the includes. So, I really think that this is a progress. I agree an XFF to XAR converter would be more interesting but my patch is ready to be applied. A XAR generator is not yet. paul
Guillaume Delhumeau <mailto:guillaume.delhumeau@xwiki.com> 23 August 2016 at 13:19 Of course, we could still change our workflow and copy/paste source contents manually. Not sure it's gonna be more effective. So I prefer the XFF approach (except that I would love having a Nested Document terminology instead of the current Space/Document one).
2016-08-23 13:11 GMT+02:00 Guillaume Delhumeau <
Guillaume Delhumeau <mailto:guillaume.delhumeau@xwiki.com> 23 August 2016 at 13:11 2016-08-23 12:36 GMT+02:00 Paul Libbrecht <paul@hoplahup.net>:
Thomas Mortagne wrote:
It takes the following IMO: * it works * it does not break any retro compatibility both are claimed. There are unit tests. The only danger of breaking anything is if dom4j does not faithfully restore the XML source. I believe this can be ignored just as it has been ignored by the implementation of transformations. (probably not enabled by default for example) No, it is enabled by default. I could replace the dom4j output by a file copy if no relevant elements are found.
Note however, that it operates on elements that were never allowed before.
Now Guillaume has a point. The issue you will quickly have as soon as you start using this is that you HAVE to modify your XAR trough filesystem because you can't edit it in XWiki and export it anymore. That is if you don't provide any tool to export this kind of XAR extension. I believe this is best practice to insure that the wiki does not insert you surprise elements such as the author name or the edit comment.
In practice, this is the only manual step. I do a diff of the modifications I have made and I revert all changes that are meaningless. But it's quick & easy using my IDE (IntelliJ). I know some developers here do the same. The only issue is when the XAR format have changed: some fields are not ordered in the same way, and it makes harder to compute the real diff.
Is this practice not corresponding at all what others are doing?
Using an XML-diff might be answering Guillaume's point... That's not completely trivial.
Paul _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
Paul Libbrecht <mailto:paul@hoplahup.net> 23 August 2016 at 12:36 Thomas Mortagne wrote:
It takes the following IMO: * it works * it does not break any retro compatibility both are claimed. There are unit tests. The only danger of breaking anything is if dom4j does not faithfully restore the XML source. I believe this can be ignored just as it has been ignored by the implementation of transformations. (probably not enabled by default for example) No, it is enabled by default. I could replace the dom4j output by a file copy if no relevant elements are found.
Note however, that it operates on elements that were never allowed before.
Now Guillaume has a point. The issue you will quickly have as soon as you start using this is that you HAVE to modify your XAR trough filesystem because you can't edit it in XWiki and export it anymore. That is if you don't provide any tool to export this kind of XAR extension. I believe this is best practice to insure that the wiki does not insert you surprise elements such as the author name or the edit comment. Is this practice not corresponding at all what others are doing?
Using an XML-diff might be answering Guillaume's point... That's not completely trivial.
Paul _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs Thomas Mortagne <mailto:thomas.mortagne@xwiki.com> 23 August 2016 at 12:28 On Tue, Aug 23, 2016 at 12:26 PM, Thomas Mortagne <thomas.mortagne@xwiki.com> wrote:
On Tue, Aug 23, 2016 at 12:17 PM, Paul Libbrecht <paul@hoplahup.net> wrote:
What I obtain after mvn package is a file my-application.xff.
I can't seem to import that. Should I be able to? Not trough XAR tools no, it's completely different format. So what needs to be done here is a converter which is a lot bigger than a packager. Which is what I indicated: you just use XFF filter (which already exist) in input and XAR filter (which also already exist) as output.
We'll see what Jean says.
In the meantime, do I understand at least that an enrichment to the xar-mojo is a more conservative approach that might have a better chance of uptake?
What does it take for me to get the patch accepted?
It takes the following IMO: * it works * it does not break any retro compatibility (probably not enabled by default for example)
Now Guillaume has a point. The issue you will quickly have as soon as you start using this is that you HAVE to modify your XAR trough filesystem because you can't edit it in XWiki and export it anymore. That is if you don't provide any tool to export this kind of XAR extension.
thanks
Paul _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs -- Thomas Mortagne
Thomas Mortagne <mailto:thomas.mortagne@xwiki.com> 23 August 2016 at 12:26 On Tue, Aug 23, 2016 at 12:17 PM, Paul Libbrecht <paul@hoplahup.net> wrote:
What I obtain after mvn package is a file my-application.xff.
I can't seem to import that. Should I be able to? Not trough XAR tools no, it's completely different format. So what needs to be done here is a converter which is a lot bigger than a packager.
Which is what I indicated: you just use XFF filter (which already exist) in input and XAR filter (which also already exist) as output.
We'll see what Jean says.
In the meantime, do I understand at least that an enrichment to the xar-mojo is a more conservative approach that might have a better chance of uptake? What does it take for me to get the patch accepted?
thanks
Paul _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
2016-08-24 20:31 GMT+02:00 Paul Libbrecht <paul@hoplahup.net>:
Guillaume,
sure, this is *doable* but it can also be done at discretion. E.g. you'd only copy from the exploded xar the other files you copied but not the source. Also, if you replace the XML files with those of the XARs, you'll just ignore the sources, since you've removed the includes.
I would be very concerned if we have wiki pages with dead code in some repository. I am very skeptical, because if we cannot rely on the standard exporter to generate the source correctly, who will use these source files? When you develop an application, do you write it in the XML page in your repository or directly in your wiki? I am a bit conservative here and I don't want to block you, but I really wonder if the advantages you mention worth the pain of being sure we don't have dead code, and manually cutting/pasting some bit of the code from a file to an other everytime you commit a page (and I do it often). Thanks, Guillaume
So, I really think that this is a progress. I agree an XFF to XAR converter would be more interesting but my patch is ready to be applied. A XAR generator is not yet.
paul
Guillaume Delhumeau <mailto:guillaume.delhumeau@xwiki.com> 23 August 2016 at 13:19 Of course, we could still change our workflow and copy/paste source contents manually. Not sure it's gonna be more effective. So I prefer the XFF approach (except that I would love having a Nested Document terminology instead of the current Space/Document one).
2016-08-23 13:11 GMT+02:00 Guillaume Delhumeau <
Guillaume Delhumeau <mailto:guillaume.delhumeau@xwiki.com> 23 August 2016 at 13:11 2016-08-23 12:36 GMT+02:00 Paul Libbrecht <paul@hoplahup.net>:
Thomas Mortagne wrote:
It takes the following IMO: * it works * it does not break any retro compatibility both are claimed. There are unit tests. The only danger of breaking anything is if dom4j does not faithfully restore the XML source. I believe this can be ignored just as it has been ignored by the implementation of transformations. (probably not enabled by default for example) No, it is enabled by default. I could replace the dom4j output by a file copy if no relevant elements are found.
Note however, that it operates on elements that were never allowed before.
Now Guillaume has a point. The issue you will quickly have as soon as you start using this is that you HAVE to modify your XAR trough filesystem because you can't edit it in XWiki and export it anymore. That is if you don't provide any tool to export this kind of XAR extension. I believe this is best practice to insure that the wiki does not insert you surprise elements such as the author name or the edit comment.
In practice, this is the only manual step. I do a diff of the modifications I have made and I revert all changes that are meaningless. But it's quick & easy using my IDE (IntelliJ). I know some developers here do the same. The only issue is when the XAR format have changed: some fields are not ordered in the same way, and it makes harder to compute the real diff.
Is this practice not corresponding at all what others are doing?
Using an XML-diff might be answering Guillaume's point... That's not completely trivial.
Paul _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
Paul Libbrecht <mailto:paul@hoplahup.net> 23 August 2016 at 12:36 Thomas Mortagne wrote:
It takes the following IMO: * it works * it does not break any retro compatibility both are claimed. There are unit tests. The only danger of breaking anything is if dom4j does not faithfully restore the XML source. I believe this can be ignored just as it has been ignored by the implementation of transformations. (probably not enabled by default for example) No, it is enabled by default. I could replace the dom4j output by a file copy if no relevant elements are found.
Note however, that it operates on elements that were never allowed before.
Now Guillaume has a point. The issue you will quickly have as soon as you start using this is that you HAVE to modify your XAR trough filesystem because you can't edit it in XWiki and export it anymore. That is if you don't provide any tool to export this kind of XAR extension. I believe this is best practice to insure that the wiki does not insert you surprise elements such as the author name or the edit comment. Is this practice not corresponding at all what others are doing?
Using an XML-diff might be answering Guillaume's point... That's not completely trivial.
Paul _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs Thomas Mortagne <mailto:thomas.mortagne@xwiki.com> 23 August 2016 at 12:28 On Tue, Aug 23, 2016 at 12:26 PM, Thomas Mortagne <thomas.mortagne@xwiki.com> wrote:
On Tue, Aug 23, 2016 at 12:17 PM, Paul Libbrecht <paul@hoplahup.net> wrote:
What I obtain after mvn package is a file my-application.xff. > I can't seem to import that. Should I be able to? Not trough XAR tools no, it's completely different format. So what needs to be done here is a converter which is a lot bigger than a packager. Which is what I indicated: you just use XFF filter (which already exist) in input and XAR filter (which also already exist) as output.
We'll see what Jean says.
In the meantime, do I understand at least that an enrichment to the xar-mojo is a more conservative approach that might have a better chance of uptake?
What does it take for me to get the patch accepted?
It takes the following IMO: * it works * it does not break any retro compatibility (probably not enabled by default for example)
Now Guillaume has a point. The issue you will quickly have as soon as you start using this is that you HAVE to modify your XAR trough filesystem because you can't edit it in XWiki and export it anymore. That is if you don't provide any tool to export this kind of XAR extension.
thanks
Paul _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs -- Thomas Mortagne
Thomas Mortagne <mailto:thomas.mortagne@xwiki.com> 23 August 2016 at 12:26 On Tue, Aug 23, 2016 at 12:17 PM, Paul Libbrecht <paul@hoplahup.net> wrote:
What I obtain after mvn package is a file my-application.xff.
I can't seem to import that. Should I be able to? Not trough XAR tools no, it's completely different format. So what needs to be done here is a converter which is a lot bigger than a packager.
Which is what I indicated: you just use XFF filter (which already exist) in input and XAR filter (which also already exist) as output.
We'll see what Jean says.
In the meantime, do I understand at least that an enrichment to the xar-mojo is a more conservative approach that might have a better chance of uptake? What does it take for me to get the patch accepted?
thanks
Paul _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Guillaume Delhumeau (guillaume.delhumeau@xwiki.com) Research & Development Engineer at XWiki SAS Committer on the XWiki.org project
Guillaume, The risk of dead code is always present at a developer. But the addition of inclusions seems to make this risk higher for you. Obviously we develop on very different wikis. Mine tends to be having several development traces, often parallel, sometimes not, and I will only reinstall if I really need it. When developing xars, you seem to be trusting that yours contain just a few. Because mine is crowdy, I will not trust a XAR export and will only pick the necessary files and/or elements form a xar export, if at all, or undergo a deep cleaning (which is typical at start). Because yours (I assume) is somewhat clean, you trust a xar export and commit it the eyes shut. Thus introducing an include that would be forgotten makes it probable for you and not for me, which is what you state. What I can offer is to add to my ready available code that the validate Mojo to check that no file that is not included in in the source tree. Useful? Offering a diff is tempting but it's a lot more complex I feel (e.g. it's better if it's three way which developers rarely all have on a single disk, one or two of them being on a versioning server). Paul Guillaume Delhumeau wrote:
I would be very concerned if we have wiki pages with dead code in some repository.
I am very skeptical, because if we cannot rely on the standard exporter to generate the source correctly, who will use these source files? When you develop an application, do you write it in the XML page in your repository or directly in your wiki?
I am a bit conservative here and I don't want to block you, but I really wonder if the advantages you mention worth the pain of being sure we don't have dead code, and manually cutting/pasting some bit of the code from a file to an other everytime you commit a page (and I do it often).
On 26 Aug 2016, at 13:24, Paul Libbrecht <paul@hoplahup.net> wrote:
Guillaume,
The risk of dead code is always present at a developer. But the addition of inclusions seems to make this risk higher for you.
Obviously we develop on very different wikis. Mine tends to be having several development traces, often parallel, sometimes not, and I will only reinstall if I really need it. When developing xars, you seem to be trusting that yours contain just a few.
Because mine is crowdy, I will not trust a XAR export and will only pick the necessary files and/or elements form a xar export, if at all, or undergo a deep cleaning (which is typical at start).
Because yours (I assume) is somewhat clean, you trust a xar export and commit it the eyes shut. Thus introducing an include that would be forgotten makes it probable for you and not for me, which is what you state.
What I can offer is to add to my ready available code that the validate Mojo to check that no file that is not included in in the source tree. Useful?
Useful but not enough IMO. This looks like just a workaround and it still means some manual action to do after you’ve export your XAR into the source tree. You still need to manually convert the document content which is XML-encoded as a non-xml encoded content in the separate file. BTW on this topic, our XAR export should really use CDATA construct and stop encoding the content. This is just a major PITA. Another idea is to offer 2 new Mojos in the XAR plugin: * mvn xar:deploy. This would push the sources into a running XWiki. The URL to be used would be defined as maven properties (and so would be the import options: overwrite, backup pack, etc). * mvn xar:fetch. This would do the opposite and it would accept 2 modes. Mode 1: it would fetch new versions of existing files in the source directory structure by fetching them from a running XWiki. Mode 2: there could be some maven property to define the name of a space it should fetch from and it would fetch all content inside that space. Now let’s come back to Paul’s proposal. The fetch operation could check the current source file in the filesystem prior to the fetching an find all xinclude and when it fetches a new version of that file (i.e. of the page), it would replace the content with the xinclude it had found and copy the content(s) in their separate file(s). This would allow not loosing the xincludes. And it would automate the whole process even more. WDYT? FTR I find Paul’s idea quite clever with XInclude. I don’t think it’s our final solution but it’s a clever one that addresses one need: that of being able to use desktop tools to edit various file types (css, js, plain text). So to conclude and while waiting for more elaborate and final solutions, I’d agree with Paul’s idea but I think we need: * Modify XWiki’s export to use CDATA instead of xml-encoding the contents. This allows to more easily and manually copy content into separate files. * Implement the 2 mojos I mentioned above which should automate nicely our process + avoid loosing xincludes. * Paul’s idea of generating a warning if there’s a file in the tree that doesn’t have a corresponding xinclude. WDYT? Thanks -Vincent
Offering a diff is tempting but it's a lot more complex I feel (e.g. it's better if it's three way which developers rarely all have on a single disk, one or two of them being on a versioning server).
Paul
Guillaume Delhumeau wrote:
I would be very concerned if we have wiki pages with dead code in some repository.
I am very skeptical, because if we cannot rely on the standard exporter to generate the source correctly, who will use these source files? When you develop an application, do you write it in the XML page in your repository or directly in your wiki?
I am a bit conservative here and I don't want to block you, but I really wonder if the advantages you mention worth the pain of being sure we don't have dead code, and manually cutting/pasting some bit of the code from a file to an other everytime you commit a page (and I do it often).
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
On 26 Aug 2016, at 18:51, Vincent Massol <vincent@massol.net> wrote:
On 26 Aug 2016, at 13:24, Paul Libbrecht <paul@hoplahup.net> wrote:
Guillaume,
The risk of dead code is always present at a developer. But the addition of inclusions seems to make this risk higher for you.
Obviously we develop on very different wikis. Mine tends to be having several development traces, often parallel, sometimes not, and I will only reinstall if I really need it. When developing xars, you seem to be trusting that yours contain just a few.
Because mine is crowdy, I will not trust a XAR export and will only pick the necessary files and/or elements form a xar export, if at all, or undergo a deep cleaning (which is typical at start).
Because yours (I assume) is somewhat clean, you trust a xar export and commit it the eyes shut. Thus introducing an include that would be forgotten makes it probable for you and not for me, which is what you state.
What I can offer is to add to my ready available code that the validate Mojo to check that no file that is not included in in the source tree. Useful?
Useful but not enough IMO. This looks like just a workaround and it still means some manual action to do after you’ve export your XAR into the source tree. You still need to manually convert the document content which is XML-encoded as a non-xml encoded content in the separate file.
BTW on this topic, our XAR export should really use CDATA construct and stop encoding the content. This is just a major PITA.
Another idea is to offer 2 new Mojos in the XAR plugin:
* mvn xar:deploy. This would push the sources into a running XWiki. The URL to be used would be defined as maven properties (and so would be the import options: overwrite, backup pack, etc).
* mvn xar:fetch. This would do the opposite and it would accept 2 modes. Mode 1: it would fetch new versions of existing files in the source directory structure by fetching them from a running XWiki. Mode 2: there could be some maven property to define the name of a space it should fetch from and it would fetch all content inside that space.
Now let’s come back to Paul’s proposal. The fetch operation could check the current source file in the filesystem prior to the fetching an find all xinclude and when it fetches a new version of that file (i.e. of the page), it would replace the content with the xinclude it had found and copy the content(s) in their separate file(s).
This would allow not loosing the xincludes. And it would automate the whole process even more.
WDYT?
FTR I find Paul’s idea quite clever with XInclude. I don’t think it’s our final solution but it’s a clever one that addresses one need: that of being able to use desktop tools to edit various file types (css, js, plain text).
So to conclude and while waiting for more elaborate and final solutions, I’d agree with Paul’s idea but I think we need:
* Modify XWiki’s export to use CDATA instead of xml-encoding the contents. This allows to more easily and manually copy content into separate files. * Implement the 2 mojos I mentioned above which should automate nicely our process + avoid loosing xincludes. * Paul’s idea of generating a warning if there’s a file in the tree that doesn’t have a corresponding xinclude.
WDYT?
Another option is to modify xar:format to always extract content + textarea properties + attachments into separate files. And agree that’s we’re ok to switch to this representation. Thanks -Vincent
Thanks -Vincent
Offering a diff is tempting but it's a lot more complex I feel (e.g. it's better if it's three way which developers rarely all have on a single disk, one or two of them being on a versioning server).
Paul
Guillaume Delhumeau wrote:
I would be very concerned if we have wiki pages with dead code in some repository.
I am very skeptical, because if we cannot rely on the standard exporter to generate the source correctly, who will use these source files? When you develop an application, do you write it in the XML page in your repository or directly in your wiki?
I am a bit conservative here and I don't want to block you, but I really wonder if the advantages you mention worth the pain of being sure we don't have dead code, and manually cutting/pasting some bit of the code from a file to an other everytime you commit a page (and I do it often).
On 26 Aug 2016, at 18:51, Vincent Massol <vincent@massol.net> wrote:
On 26 Aug 2016, at 13:24, Paul Libbrecht <paul@hoplahup.net> wrote:
Guillaume,
The risk of dead code is always present at a developer. But the addition of inclusions seems to make this risk higher for you.
Obviously we develop on very different wikis. Mine tends to be having several development traces, often parallel, sometimes not, and I will only reinstall if I really need it. When developing xars, you seem to be trusting that yours contain just a few.
Because mine is crowdy, I will not trust a XAR export and will only pick the necessary files and/or elements form a xar export, if at all, or undergo a deep cleaning (which is typical at start).
Because yours (I assume) is somewhat clean, you trust a xar export and commit it the eyes shut. Thus introducing an include that would be forgotten makes it probable for you and not for me, which is what you state.
What I can offer is to add to my ready available code that the validate Mojo to check that no file that is not included in in the source tree. Useful?
Useful but not enough IMO. This looks like just a workaround and it still means some manual action to do after you’ve export your XAR into the source tree. You still need to manually convert the document content which is XML-encoded as a non-xml encoded content in the separate file.
BTW on this topic, our XAR export should really use CDATA construct and stop encoding the content. This is just a major PITA.
Another idea is to offer 2 new Mojos in the XAR plugin:
* mvn xar:deploy. This would push the sources into a running XWiki. The URL to be used would be defined as maven properties (and so would be the import options: overwrite, backup pack, etc).
* mvn xar:fetch. This would do the opposite and it would accept 2 modes. Mode 1: it would fetch new versions of existing files in the source directory structure by fetching them from a running XWiki. Mode 2: there could be some maven property to define the name of a space it should fetch from and it would fetch all content inside that space.
Now let’s come back to Paul’s proposal. The fetch operation could check the current source file in the filesystem prior to the fetching an find all xinclude and when it fetches a new version of that file (i.e. of the page), it would replace the content with the xinclude it had found and copy the content(s) in their separate file(s).
This would allow not loosing the xincludes. And it would automate the whole process even more.
WDYT?
FTR I find Paul’s idea quite clever with XInclude. I don’t think it’s our final solution but it’s a clever one that addresses one need: that of being able to use desktop tools to edit various file types (css, js, plain text).
So to conclude and while waiting for more elaborate and final solutions, I’d agree with Paul’s idea but I think we need:
* Modify XWiki’s export to use CDATA instead of xml-encoding the contents. This allows to more easily and manually copy content into separate files.
BTW on a related note, I was wondering if XInclude allows including non-XML encoded content. I’ve found https://msdn.microsoft.com/en-us/library/aa302291.aspx#xinc_topic4 which seems to indicate it’s supported (with parse=text and by using the encoding attribute too). Thanks -Vincent
* Implement the 2 mojos I mentioned above which should automate nicely our process + avoid loosing xincludes. * Paul’s idea of generating a warning if there’s a file in the tree that doesn’t have a corresponding xinclude.
WDYT?
Thanks -Vincent
Offering a diff is tempting but it's a lot more complex I feel (e.g. it's better if it's three way which developers rarely all have on a single disk, one or two of them being on a versioning server).
Paul
Guillaume Delhumeau wrote:
I would be very concerned if we have wiki pages with dead code in some repository.
I am very skeptical, because if we cannot rely on the standard exporter to generate the source correctly, who will use these source files? When you develop an application, do you write it in the XML page in your repository or directly in your wiki?
I am a bit conservative here and I don't want to block you, but I really wonder if the advantages you mention worth the pain of being sure we don't have dead code, and manually cutting/pasting some bit of the code from a file to an other everytime you commit a page (and I do it often).
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
Vincent Massol wrote:
* Modify XWiki’s export to use CDATA instead of xml-encoding the contents. This allows to more easily and manually copy content into separate files.
BTW on a related note, I was wondering if XInclude allows including non-XML encoded content. I’ve found https://msdn.microsoft.com/en-us/library/aa302291.aspx#xinc_topic4 which seems to indicate it’s supported (with parse=text and by using the encoding attribute too). Sure it does and this is unit-tested! See the xincludeTTest.xml in here
http://direct.hoplahup.net/tmp/xwiki-commons-tool-xar-plugin/src/test/resour... it is tested to match xincludeTProof.xml. However, it does not do base64 conversion... Hence I've introduced two elements textInclude (does the same, just no namespace fuss) and binaryInclude) includes the attachment converted to base64. Paul
On Tue, Aug 23, 2016 at 11:35 AM, Vincent Massol <vincent@massol.net> wrote:
On 23 Aug 2016, at 11:15, Vincent Massol <vincent@massol.net> wrote:
On 23 Aug 2016, at 11:13, Paul Libbrecht <paul@hoplahup.net> wrote:
Let's ask things differently: what's missing XFF for others to use?
I think it’s just missing a maven plugin to use it. All the other parts are ready.
Actually just noticed that Jean has started one: https://github.com/xwiki-contrib/api-xff/tree/master/xff-maven
I don’t know its status though.
So what needs to be done is try the full chain and see what works/what doesn’t.
Unless jean could tell us what’s the status? (I’ll ping him on skype).
This plugin has nothing to do with XAR AFAIK. It just zip a XFF package.
Thanks -Vincent
Thanks -Vincent
paul
Vincent Massol <mailto:vincent@massol.net> 23 August 2016 at 11:01 The format seems to be defined here: http://extensions.xwiki.org/xwiki/bin/view/Extension/XFF+filter#HFormat
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne
Hello Paul. I think it's an interesting idea (and I know this is not the first proposal in that way). But do you also consider the opposite use-case: editing some code in the wiki, and then export a XAR containing the metadata in an XML and the content in a plain text file? Otherwise, it makes it harder to fix some code in the wiki directly before committing it in the repository. Thanks, 2016-08-22 15:59 GMT+02:00 Paul Libbrecht <paul@hoplahup.net>:
Hello fellow developers,
many of you are developing XAR-packaged resources. The exchangeability of these archives is one of the precious aspects of XWiki, I believe.
However, if one see all the many GitHub-contributed xar packages, the source is not there: it's in a Wiki with which one would "open it". A pity since much of the content is code.
I'd like to propose to allow XARs to be built form source code which is true source that one edits. Code files with their proper syntax coloring, editors, auto-completion, search. Media-files that are edited with our tool of choice by a simple double-click and not a complex pull, edit, and replace.
Thus, my proposal is to allow inclusions within the XML files. Inclusions should be of type: - text to include page content or object properties (e.g. css of cssx objects) - attachments to include attached objects
My current setup is documented here:http://jira.xwiki.org/ browse/XWIKI-13643 It does binaryInclude to generate base64 and textInclude to generate XML-text. Thus, I can advertise a github project which is a macro and whose complex function is in a velocity file that is in a .vm file. However, I do not need to explode everything, e.g. the text-content describing the macro or (in this case too small) translation set.
What do you think? Should I rather generate complete attachment elements (name, length, modif date, ...) instead of doing a binaryInclude that just generates base64 ? Is fetching files for inclusion beyond the boundaries of the project of any use? (it is currently prohibited for safety)
thanks in advance.
Paul _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Guillaume Delhumeau (guillaume.delhumeau@xwiki.com) Research & Development Engineer at XWiki SAS Committer on the XWiki.org project
To be extra-clear, let me explain my workflow when I design an application in XWiki. - First, I write my code directly in an instance of XWiki. - When it's ready, I export all the pages related to my application in a XAR. - Then, I uncompress my XAR inside my repository. - I clean the XML files using mvn xar:format. - I commit. And when I fix something, I do it directly in the wiki, then export the page in a XAR, uncompress it in my repository, clean the xml files manually (reverting the changed dates, etc...). What I don't want is to be forced to manually remove the source code from the XML to put it in the other file every time. Thanks, 2016-08-23 10:45 GMT+02:00 Guillaume Delhumeau < guillaume.delhumeau@xwiki.com>:
Hello Paul.
I think it's an interesting idea (and I know this is not the first proposal in that way).
But do you also consider the opposite use-case: editing some code in the wiki, and then export a XAR containing the metadata in an XML and the content in a plain text file?
Otherwise, it makes it harder to fix some code in the wiki directly before committing it in the repository.
Thanks,
2016-08-22 15:59 GMT+02:00 Paul Libbrecht <paul@hoplahup.net>:
Hello fellow developers,
many of you are developing XAR-packaged resources. The exchangeability of these archives is one of the precious aspects of XWiki, I believe.
However, if one see all the many GitHub-contributed xar packages, the source is not there: it's in a Wiki with which one would "open it". A pity since much of the content is code.
I'd like to propose to allow XARs to be built form source code which is true source that one edits. Code files with their proper syntax coloring, editors, auto-completion, search. Media-files that are edited with our tool of choice by a simple double-click and not a complex pull, edit, and replace.
Thus, my proposal is to allow inclusions within the XML files. Inclusions should be of type: - text to include page content or object properties (e.g. css of cssx objects) - attachments to include attached objects
My current setup is documented here:http://jira.xwiki.org/bro wse/XWIKI-13643 It does binaryInclude to generate base64 and textInclude to generate XML-text. Thus, I can advertise a github project which is a macro and whose complex function is in a velocity file that is in a .vm file. However, I do not need to explode everything, e.g. the text-content describing the macro or (in this case too small) translation set.
What do you think? Should I rather generate complete attachment elements (name, length, modif date, ...) instead of doing a binaryInclude that just generates base64 ? Is fetching files for inclusion beyond the boundaries of the project of any use? (it is currently prohibited for safety)
thanks in advance.
Paul _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Guillaume Delhumeau (guillaume.delhumeau@xwiki.com) Research & Development Engineer at XWiki SAS Committer on the XWiki.org project
-- Guillaume Delhumeau (guillaume.delhumeau@xwiki.com) Research & Development Engineer at XWiki SAS Committer on the XWiki.org project
Maybe what we need is a xar diff function ?? (compare my xar's source repo to the content of a wiki and say what needs to be changed) paul
Guillaume Delhumeau <mailto:guillaume.delhumeau@xwiki.com> 23 August 2016 at 10:51 To be extra-clear, let me explain my workflow when I design an application in XWiki.
- First, I write my code directly in an instance of XWiki. - When it's ready, I export all the pages related to my application in a XAR. - Then, I uncompress my XAR inside my repository. - I clean the XML files using mvn xar:format. - I commit.
And when I fix something, I do it directly in the wiki, then export the page in a XAR, uncompress it in my repository, clean the xml files manually (reverting the changed dates, etc...).
What I don't want is to be forced to manually remove the source code from the XML to put it in the other file every time.
Thanks,
But do you also consider the opposite use-case: editing some code in the wiki, and then export a XAR containing the metadata in an XML and the content in a plain text file? Otherwise, it makes it harder to fix some code in the wiki directly before committing it in the repository. No. Not for this case. I guess it could be implemented by detecting which things are outside and which are inside but it's not really trivial.
As far as I know, fetching the xar form the wiki really is a first step which you do not do on a regular basis, you then push adjustments to the wiki which you have edited in the source directory (e.g. using copy and paste or attachments' uploader), then do an upload as a xar to make sure it is clean. The problem with fetching from the wiki is that it always inserts more information than wanted. E.g. a creator name, comments, ... Whereas creating the xar from source can be tailored to not contain any of this information (and part of that is checked by the xar mojo). Am I wrong? Paul
Hi Paul, I’ve finally taken the time to read this discussion (a bit quickly for the last mail though, I admit ;)). Let me give my POV: * What I understand is that you’re not proposing to change the XAR format but to introduce some tooling to generate a XAR from a set of source files, at build time. Correct? * IMO we should not touch the existing XAR plugin. This is current the standard way to generate XAR files and we need to keep, if only for backward compatibility with existing sources. * IMO you should develop your tool as a new maven plugin in xwiki-contrib and use it on some contrib projects to validate and show to others how it works. Of course you can depend on the existing XAR plugin if you need tor reuse some code (it’s even possible to introduce a new api module if there’s an important need to share common code). * FTR this is exactly what Jean Simard as done with the XFF tools (https://github.com/xwiki-contrib/api-xff). He’s done is separately to validate it and show it. We need to know where we stand on XFF since it’s currently the most advanced POC that we have on introducing a new format for storing wiki pages in the SCM. It also supports both directions: from source to XAR and from wiki pages to source. I’m planning to check what’s missing in that progress to be able to use it (not much IMO). * If you were to develop your maven plugin, you would just compete with the XFF code. This is fine, although I’d have preferred that we spend our time finishing the XFF work. It’s your time so your choice though. In the future, we’ll be able to choose which solution we want to use between the 3 that will exist (current XAR plugin, XFF solution and your solution). * FTR Fabio had also worked on this topic in the past, see http://design.xwiki.org/xwiki/bin/view/Design/DirectoryStructureforXWikiAppl.... Caleb has also worked on something related: https://github.com/xwiki-contrib/xwiki-tools-node So for me the important part is for you to develop your solution in a separate module in xwiki-contrib as a demonstator and possibly start using it on some contrib project to validate it. The alternative is to work on XFF and finish it. Personally I’m not a big fan at all of modifying the existing XAR plugin for various reasons: * We don’t know yet what solution we’ll choose in the future. At this point in time I even have a preference for XFF. So it wouldn’t be fair and nice to modify the existing XAR plugin with your solution and then have to remove the code if we don’ want it in the future. * It will be much simpler for you to develop your own project in xwiki-contrib since you’ll have commit access (which you won’t have in xwiki-commons). WDYT? Thanks -Vincent
On 22 Aug 2016, at 15:59, Paul Libbrecht <paul@hoplahup.net> wrote:
Hello fellow developers,
many of you are developing XAR-packaged resources. The exchangeability of these archives is one of the precious aspects of XWiki, I believe.
However, if one see all the many GitHub-contributed xar packages, the source is not there: it's in a Wiki with which one would "open it". A pity since much of the content is code.
I'd like to propose to allow XARs to be built form source code which is true source that one edits. Code files with their proper syntax coloring, editors, auto-completion, search. Media-files that are edited with our tool of choice by a simple double-click and not a complex pull, edit, and replace.
Thus, my proposal is to allow inclusions within the XML files. Inclusions should be of type: - text to include page content or object properties (e.g. css of cssx objects) - attachments to include attached objects
My current setup is documented here:http://jira.xwiki.org/browse/XWIKI-13643 It does binaryInclude to generate base64 and textInclude to generate XML-text. Thus, I can advertise a github project which is a macro and whose complex function is in a velocity file that is in a .vm file. However, I do not need to explode everything, e.g. the text-content describing the macro or (in this case too small) translation set.
What do you think? Should I rather generate complete attachment elements (name, length, modif date, ...) instead of doing a binaryInclude that just generates base64 ? Is fetching files for inclusion beyond the boundaries of the project of any use? (it is currently prohibited for safety)
thanks in advance.
Paul
Hello VIncent, > So for me the important part is for you to develop your solution in a separate module in xwiki-contrib as a demonstator and possibly start using it on some contrib project to validate it. The alternative is to work on XFF and finish it. Well. That brings the question back: how many are using it? Obviously, I see almost no-one. As long as: - it is not complete - it is not used by developers big time ... it will not be considered useful. So... requesting to "go away" is very close to what you're asking, I feel. In the contrary, I believe that changing the XAR plugin in a fully conservative way is the way to go if one wants incremental progress. Can it be that you did not notice, it is fully backwards compatible? > Personally I’m not a big fan at all of modifying the existing XAR plugin for various reasons: > * We don’t know yet what solution we’ll choose in the future. At this point in time I even have a preference for XFF. So it wouldn’t be fair and nice to modify the existing XAR plugin with your solution and then have to remove the code if we don’ want it in the future. But obviously the time has not been invested about it. All issues are closed for XFF. > * It will be much simpler for you to develop your own project in xwiki-contrib since you’ll have commit access (which you won’t have in xwiki-commons). That's a moot criterion. My code is really not a big change and also not very high risk. Paul
On 26 Aug 2016, at 14:03, Paul Libbrecht <paul@hoplahup.net> wrote:
Hello VIncent,
So for me the important part is for you to develop your solution in a separate module in xwiki-contrib as a demonstator and possibly start using it on some contrib project to validate it. The alternative is to work on XFF and finish it. Well. That brings the question back: how many are using it? Obviously, I see almost no-one.
ofc since it isn’t finished and has never been promoted and the xwiki product team has not even decided whether they’d like to use it or not for the apps in the platform repo...
As long as: - it is not complete - it is not used by developers big time ... it will not be considered useful. So... requesting to "go away" is very close to what you're asking, I feel.
Too bad you’re seeing it this way but I was actually trying to help you, so that you could progress (not be blocked) and show something (to convince people). Now if your point is that either your tool should be used by the xwiki dev team or you don’t want to spend time working on it, then it requires a lot more discussions and it's required that it solves the issue of moving from wiki pages to sources which you haven’t addressed (but which XFF has). To repeat myself, ATM, with my hat of xwiki dev team member, I prefer the XFF solution because 1) it’s a complete solution and 2) it’s following the direction we want to take (Filters). We still need to assess how much work remain but I’m confident it’s not too much.
In the contrary, I believe that changing the XAR plugin in a fully conservative way is the way to go if one wants incremental progress.
As soon as the code is in, it’s the work of the xwiki dev team to support it even though we may not even be using it. What’s the point? Honestly I don’t understand why you cannot put your code in xwiki-contrib under a different maven plugin than the XAR one. Name it XAR2 if you want. As for using it in a project it’s a simple as declaring 3 lines so that cannot be the issue.
Can it be that you did not notice, it is fully backwards compatible?
What I haven’t fully understood is how would the sources of wiki pages look like in the file system with your proposal. Do you have an example? Can I test it somewhere so that I can fully understand your technical solution?
Personally I’m not a big fan at all of modifying the existing XAR plugin for various reasons: * We don’t know yet what solution we’ll choose in the future. At this point in time I even have a preference for XFF. So it wouldn’t be fair and nice to modify the existing XAR plugin with your solution and then have to remove the code if we don’ want it in the future. But obviously the time has not been invested about it. All issues are closed for XFF.
So let’s do it! :) It hasn’t been invested more because the xwiki dev team doesn’t have such a pressing need compared to other stuff, that’s all. Now I’ve personally been wanting to review it, make it work and make a proposal on the list about using it for the platform github repo. Now there are still problems probably. For example even though it supports exporting, it’s not convenient unless it’s bundled by default (since otherwise you’d need to install the extension everytime when you’re in dev mode, which is a pain). So we’d need to agree we want to bundle it by default for example.
* It will be much simpler for you to develop your own project in xwiki-contrib since you’ll have commit access (which you won’t have in xwiki-commons). That's a moot criterion. My code is really not a big change and also not very high risk.
What I fail to understand is why you don’t want to commit your code in contrib. XWiki is a platform with extensions. You don’t need to be in the core to be able to contribute an extension to XWiki (XAR plugin is core). Thanks -Vincent
Paul
Vincent Massol wrote:
Can I test it somewhere so that I can fully understand your technical solution? http://extensions.xwiki.org/xwiki/bin/view/Extension/DownloadAttachments/ is an example source with it. This can also be browsed at:
http://direct.hoplahup.net/tmp/DownloadAttachments/src/main/resources/Macros... where DownloadAttachments.xml includes DownloadAttachments_content.groovy. I guess that the way to test it is to build and install the project I have (which would be 8.3-SNAPHOT): http://direct.hoplahup.net/tmp/xwiki-commons-tool-xar-plugin-8.3-SNAPSHOT-po... (which you'd obtain by applying the patch I've put in the jira).
What I fail to understand is why you don’t want to commit your code in contrib. XWiki is a platform with extensions. You don’t need to be in the core to be able to contribute an extension to XWiki (XAR plugin is core). I'd like to avoid forking. This creates niches. My laptop is full of such attempts. Creating a github fork, a maven polxar or something like this is fairly equivalent.
Paul
On 26 Aug 2016, at 16:04, Paul Libbrecht <paul@hoplahup.net> wrote:
Vincent Massol wrote:
Can I test it somewhere so that I can fully understand your technical solution? http://extensions.xwiki.org/xwiki/bin/view/Extension/DownloadAttachments/ is an example source with it. This can also be browsed at:
http://direct.hoplahup.net/tmp/DownloadAttachments/src/main/resources/Macros... where DownloadAttachments.xml includes DownloadAttachments_content.groovy.
<note on FF> Just checked http://extensions.xwiki.org/xwiki/bin/view/Extension/XFF+filter#HFormat and I was surprised to see that Jean has not defined a different file for the content and there’s still XML. I remember seeing other proposals int he past where there was not a single line of XML (I think I even proposed some format on the list a long time ago based on json for the metadata since that’s much simpler to write). It does separate CSS and JS and attachments but it should also do the same for content. We’d need to work on that. </note on FF> In my vision of a nice filesystem format, you’d have a multiple level directory structure (to represent nested pages) and a lot of conventions (such as having a file name content.txt in each directory to represent that content and an optional meta.json file to represent the metadata: authors, language, etc) and some reserved directory names for xobjects, etc.
I guess that the way to test it is to build and install the project I have (which would be 8.3-SNAPHOT):
http://direct.hoplahup.net/tmp/xwiki-commons-tool-xar-plugin-8.3-SNAPSHOT-po... (which you'd obtain by applying the patch I've put in the jira).
What I fail to understand is why you don’t want to commit your code in contrib. XWiki is a platform with extensions. You don’t need to be in the core to be able to contribute an extension to XWiki (XAR plugin is core). I'd like to avoid forking. This creates niches. My laptop is full of such attempts. Creating a github fork, a maven polxar or something like this is fairly equivalent.
Ok so if I understand correctly what you’re proposing is to keep the XML files and just use manual XInclude for separating out the content and any other part that a dev wants to separate (textarea properties, etc). So it would be a convention to use xinclude or not and not something enforced. Meaning that it’s up to the developers to use an xinclude or not. Indeed, for this part I wouldn't see a real issue in adding support for this in the existing XAR mojo since it’s very lightweight and is just an optional thing. Now we still wouldn’t be able to use it in the xwiki dev team because we need a round-trip solution (we need to be able to convert a wiki page into that structure). This is what Guillaume was mentioning. So we need to solve that first or it won’t have much of a future (and it could be argued that there’s little rationale for putting this code in commons if it’s not used). <side note> If we introduce a nice filesystem format, it will be tempting to make modifications to it directly and commit. This introduces the risk of breaking things from what we do now. Right now we also ensure it works first because we make the modifications in the wiki - ie in our execution environment - and only after we export. One way to circumvent this will be to develop XAR unit testing or functional testing a lot more. Right now our process at least enforces a manual testing... </side note> Thanks -Vincent
Paul
Vincent Massol wrote:
Now we still wouldn’t be able to use it in the xwiki dev team because we need a round-trip solution (we need to be able to convert a wiki page into that structure). This is what Guillaume was mentioning.
So we need to solve that first or it won’t have much of a future (and it could be argued that there’s little rationale for putting this code in commons if it’s not used). Is the warning proposal made to Guillaume sufficient?
Paul
Now we still wouldn’t be able to use it in the xwiki dev team because we need a round-trip solution (we need to be able to convert a wiki page into that structure). This is what Guillaume was mentioning.
So we need to solve that first or it won’t have much of a future (and it could be argued that there’s little rationale for putting this code in commons if it’s not used).
Is the warning proposal made to Guillaume sufficient? And... which team do you mean? The product team? All employees? All developers? The client team?
Paul
On 26 Aug 2016, at 17:14, Paul Libbrecht <paul@hoplahup.net> wrote:
Now we still wouldn’t be able to use it in the xwiki dev team because we need a round-trip solution (we need to be able to convert a wiki page into that structure). This is what Guillaume was mentioning.
So we need to solve that first or it won’t have much of a future (and it could be argued that there’s little rationale for putting this code in commons if it’s not used).
Is the warning proposal made to Guillaume sufficient? And... which team do you mean? The product team? All employees? All developers? The client team?
None of those! :) (btw all of these have no meaning on this devs list…) “ Now we still wouldn’t be able to use it in the xwiki dev team because we need a round-trip solution (we need to be able to convert a wiki page into that structure). This is what Guillaume was mentioning. So we need to solve that first or it won’t have much of a future (and it could be argued that there’s little rationale for putting this code in commons if it’s not used). “ team = "xwiki dev team “ == committers of the xwiki github organization == the committers of where the XAR plugin is located (in xwiki-commons in the xwiki github organization). Thanks -Vincent
Paul _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
On 26 Aug 2016, at 13:50, Vincent Massol <vincent@massol.net> wrote:
Hi Paul,
I’ve finally taken the time to read this discussion (a bit quickly for the last mail though, I admit ;)).
Let me give my POV:
* What I understand is that you’re not proposing to change the XAR format but to introduce some tooling to generate a XAR from a set of source files, at build time. Correct?
* IMO we should not touch the existing XAR plugin. This is current the standard way to generate XAR files and we need to keep, if only for backward compatibility with existing sources.
* IMO you should develop your tool as a new maven plugin in xwiki-contrib and use it on some contrib projects to validate and show to others how it works. Of course you can depend on the existing XAR plugin if you need tor reuse some code (it’s even possible to introduce a new api module if there’s an important need to share common code).
* FTR this is exactly what Jean Simard as done with the XFF tools (https://github.com/xwiki-contrib/api-xff). He’s done is separately to validate it and show it. We need to know where we stand on XFF since it’s currently the most advanced POC that we have on introducing a new format for storing wiki pages in the SCM. It also supports both directions: from source to XAR and from wiki pages to source. I’m planning to check what’s missing in that progress to be able to use it (not much IMO).
So I’ve posted a quick analysis of the XFF module here: http://markmail.org/message/2pqcyfflxhcp3udh In short: * We should be able to import an XFF structure into a running XWiki * It doesn’t seem that the round tripping is implemented and I don’t think we can export from a running XWiki into an XFF structure. * The XFF format needs to be reviewed Thanks -Vincent
* If you were to develop your maven plugin, you would just compete with the XFF code. This is fine, although I’d have preferred that we spend our time finishing the XFF work. It’s your time so your choice though. In the future, we’ll be able to choose which solution we want to use between the 3 that will exist (current XAR plugin, XFF solution and your solution).
* FTR Fabio had also worked on this topic in the past, see http://design.xwiki.org/xwiki/bin/view/Design/DirectoryStructureforXWikiAppl.... Caleb has also worked on something related: https://github.com/xwiki-contrib/xwiki-tools-node
So for me the important part is for you to develop your solution in a separate module in xwiki-contrib as a demonstator and possibly start using it on some contrib project to validate it. The alternative is to work on XFF and finish it.
Personally I’m not a big fan at all of modifying the existing XAR plugin for various reasons: * We don’t know yet what solution we’ll choose in the future. At this point in time I even have a preference for XFF. So it wouldn’t be fair and nice to modify the existing XAR plugin with your solution and then have to remove the code if we don’ want it in the future. * It will be much simpler for you to develop your own project in xwiki-contrib since you’ll have commit access (which you won’t have in xwiki-commons).
WDYT?
Thanks -Vincent
On 22 Aug 2016, at 15:59, Paul Libbrecht <paul@hoplahup.net> wrote:
Hello fellow developers,
many of you are developing XAR-packaged resources. The exchangeability of these archives is one of the precious aspects of XWiki, I believe.
However, if one see all the many GitHub-contributed xar packages, the source is not there: it's in a Wiki with which one would "open it". A pity since much of the content is code.
I'd like to propose to allow XARs to be built form source code which is true source that one edits. Code files with their proper syntax coloring, editors, auto-completion, search. Media-files that are edited with our tool of choice by a simple double-click and not a complex pull, edit, and replace.
Thus, my proposal is to allow inclusions within the XML files. Inclusions should be of type: - text to include page content or object properties (e.g. css of cssx objects) - attachments to include attached objects
My current setup is documented here:http://jira.xwiki.org/browse/XWIKI-13643 It does binaryInclude to generate base64 and textInclude to generate XML-text. Thus, I can advertise a github project which is a macro and whose complex function is in a velocity file that is in a .vm file. However, I do not need to explode everything, e.g. the text-content describing the macro or (in this case too small) translation set.
What do you think? Should I rather generate complete attachment elements (name, length, modif date, ...) instead of doing a binaryInclude that just generates base64 ? Is fetching files for inclusion beyond the boundaries of the project of any use? (it is currently prohibited for safety)
thanks in advance.
Paul
On 26 Aug 2016, at 13:50, Vincent Massol <vincent@massol.net> wrote:
Hi Paul,
I’ve finally taken the time to read this discussion (a bit quickly for the last mail though, I admit ;)).
Let me give my POV:
* What I understand is that you’re not proposing to change the XAR format but to introduce some tooling to generate a XAR from a set of source files, at build time. Correct?
* IMO we should not touch the existing XAR plugin. This is current the standard way to generate XAR files and we need to keep, if only for backward compatibility with existing sources.
* IMO you should develop your tool as a new maven plugin in xwiki-contrib and use it on some contrib projects to validate and show to others how it works. Of course you can depend on the existing XAR plugin if you need tor reuse some code (it’s even possible to introduce a new api module if there’s an important need to share common code).
* FTR this is exactly what Jean Simard as done with the XFF tools (https://github.com/xwiki-contrib/api-xff). He’s done is separately to validate it and show it. We need to know where we stand on XFF since it’s currently the most advanced POC that we have on introducing a new format for storing wiki pages in the SCM. It also supports both directions: from source to XAR and from wiki pages to source. I’m planning to check what’s missing in that progress to be able to use it (not much IMO).
* If you were to develop your maven plugin, you would just compete with the XFF code. This is fine, although I’d have preferred that we spend our time finishing the XFF work. It’s your time so your choice though. In the future, we’ll be able to choose which solution we want to use between the 3 that will exist (current XAR plugin, XFF solution and your solution).
* FTR Fabio had also worked on this topic in the past, see http://design.xwiki.org/xwiki/bin/view/Design/DirectoryStructureforXWikiAppl.... Caleb has also worked on something related: https://github.com/xwiki-contrib/xwiki-tools-node
More notes: - Caleb’s solution is working and is being used for some projects. For example: https://github.com/xwiki-contrib/realtime-wysiwyg/tree/master/src/xwiki Cons of Caleb solutions are listed at http://design.xwiki.org/xwiki/bin/view/Design/DirectoryStructureforXWikiAppl.... Main issue points IMO: * There is no automated way to create this structure from code in the Wiki (TODO) • Still lacking a Maven job for building extensions (TODO). Requires nodejs to build. - There’s also work done by Fabio here that I had missed: https://github.com/fmancinelli/xwikifs. Fabio has chosen YAML, see https://github.com/fmancinelli/xwikifs/tree/master/xwikifs-maven-plugin/src/... for an example. However I don’t know what is its status. There seems to be a Maven XAR mojo written to generate a XAR file. So we have plenty of stuff that have been tried but none has been finished and none are addressing the full needs. To summarize, so far I’ve found: * Caleb’s nodejs tool: https://github.com/xwiki-contrib/xwiki-tools-node * Fabio’s xwikifs: https://github.com/fmancinelli/xwikifs * Jean’s XFF: https://github.com/xwiki-contrib/api-xff * Paul’s XInclude extension to the XAR plugin: http://jira.xwiki.org/browse/XWIKI-13643 They all seem to be missing the ability to export wiki pages from a running wiki into the source filesystem. Thanks -Vincent
So for me the important part is for you to develop your solution in a separate module in xwiki-contrib as a demonstator and possibly start using it on some contrib project to validate it. The alternative is to work on XFF and finish it.
Personally I’m not a big fan at all of modifying the existing XAR plugin for various reasons: * We don’t know yet what solution we’ll choose in the future. At this point in time I even have a preference for XFF. So it wouldn’t be fair and nice to modify the existing XAR plugin with your solution and then have to remove the code if we don’ want it in the future. * It will be much simpler for you to develop your own project in xwiki-contrib since you’ll have commit access (which you won’t have in xwiki-commons).
WDYT?
Thanks -Vincent
On 22 Aug 2016, at 15:59, Paul Libbrecht <paul@hoplahup.net> wrote:
Hello fellow developers,
many of you are developing XAR-packaged resources. The exchangeability of these archives is one of the precious aspects of XWiki, I believe.
However, if one see all the many GitHub-contributed xar packages, the source is not there: it's in a Wiki with which one would "open it". A pity since much of the content is code.
I'd like to propose to allow XARs to be built form source code which is true source that one edits. Code files with their proper syntax coloring, editors, auto-completion, search. Media-files that are edited with our tool of choice by a simple double-click and not a complex pull, edit, and replace.
Thus, my proposal is to allow inclusions within the XML files. Inclusions should be of type: - text to include page content or object properties (e.g. css of cssx objects) - attachments to include attached objects
My current setup is documented here:http://jira.xwiki.org/browse/XWIKI-13643 It does binaryInclude to generate base64 and textInclude to generate XML-text. Thus, I can advertise a github project which is a macro and whose complex function is in a velocity file that is in a .vm file. However, I do not need to explode everything, e.g. the text-content describing the macro or (in this case too small) translation set.
What do you think? Should I rather generate complete attachment elements (name, length, modif date, ...) instead of doing a binaryInclude that just generates base64 ? Is fetching files for inclusion beyond the boundaries of the project of any use? (it is currently prohibited for safety)
thanks in advance.
Paul
devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
On 26 Aug 2016, at 18:33, Vincent Massol <vincent@massol.net> wrote:
On 26 Aug 2016, at 13:50, Vincent Massol <vincent@massol.net> wrote:
Hi Paul,
I’ve finally taken the time to read this discussion (a bit quickly for the last mail though, I admit ;)).
Let me give my POV:
* What I understand is that you’re not proposing to change the XAR format but to introduce some tooling to generate a XAR from a set of source files, at build time. Correct?
* IMO we should not touch the existing XAR plugin. This is current the standard way to generate XAR files and we need to keep, if only for backward compatibility with existing sources.
* IMO you should develop your tool as a new maven plugin in xwiki-contrib and use it on some contrib projects to validate and show to others how it works. Of course you can depend on the existing XAR plugin if you need tor reuse some code (it’s even possible to introduce a new api module if there’s an important need to share common code).
* FTR this is exactly what Jean Simard as done with the XFF tools (https://github.com/xwiki-contrib/api-xff). He’s done is separately to validate it and show it. We need to know where we stand on XFF since it’s currently the most advanced POC that we have on introducing a new format for storing wiki pages in the SCM. It also supports both directions: from source to XAR and from wiki pages to source. I’m planning to check what’s missing in that progress to be able to use it (not much IMO).
* If you were to develop your maven plugin, you would just compete with the XFF code. This is fine, although I’d have preferred that we spend our time finishing the XFF work. It’s your time so your choice though. In the future, we’ll be able to choose which solution we want to use between the 3 that will exist (current XAR plugin, XFF solution and your solution).
* FTR Fabio had also worked on this topic in the past, see http://design.xwiki.org/xwiki/bin/view/Design/DirectoryStructureforXWikiAppl.... Caleb has also worked on something related: https://github.com/xwiki-contrib/xwiki-tools-node
More notes:
- Caleb’s solution is working and is being used for some projects. For example: https://github.com/xwiki-contrib/realtime-wysiwyg/tree/master/src/xwiki
Cons of Caleb solutions are listed at http://design.xwiki.org/xwiki/bin/view/Design/DirectoryStructureforXWikiAppl.... Main issue points IMO: * There is no automated way to create this structure from code in the Wiki (TODO) • Still lacking a Maven job for building extensions (TODO). Requires nodejs to build.
- There’s also work done by Fabio here that I had missed: https://github.com/fmancinelli/xwikifs. Fabio has chosen YAML, see https://github.com/fmancinelli/xwikifs/tree/master/xwikifs-maven-plugin/src/... an example. However I don’t know what is its status. There seems to be a Maven XAR mojo written to generate a XAR file.
FTR here’s what Paul was saying in 2013 concerning xwikifs: http://markmail.org/message/az2mh65c6xuc47bp :) Thanks -Vincent
So we have plenty of stuff that have been tried but none has been finished and none are addressing the full needs. To summarize, so far I’ve found:
* Caleb’s nodejs tool: https://github.com/xwiki-contrib/xwiki-tools-node * Fabio’s xwikifs: https://github.com/fmancinelli/xwikifs * Jean’s XFF: https://github.com/xwiki-contrib/api-xff * Paul’s XInclude extension to the XAR plugin: http://jira.xwiki.org/browse/XWIKI-13643
They all seem to be missing the ability to export wiki pages from a running wiki into the source filesystem.
Thanks -Vincent
So for me the important part is for you to develop your solution in a separate module in xwiki-contrib as a demonstator and possibly start using it on some contrib project to validate it. The alternative is to work on XFF and finish it.
Personally I’m not a big fan at all of modifying the existing XAR plugin for various reasons: * We don’t know yet what solution we’ll choose in the future. At this point in time I even have a preference for XFF. So it wouldn’t be fair and nice to modify the existing XAR plugin with your solution and then have to remove the code if we don’ want it in the future. * It will be much simpler for you to develop your own project in xwiki-contrib since you’ll have commit access (which you won’t have in xwiki-commons).
WDYT?
Thanks -Vincent
On 22 Aug 2016, at 15:59, Paul Libbrecht <paul@hoplahup.net> wrote:
Hello fellow developers,
many of you are developing XAR-packaged resources. The exchangeability of these archives is one of the precious aspects of XWiki, I believe.
However, if one see all the many GitHub-contributed xar packages, the source is not there: it's in a Wiki with which one would "open it". A pity since much of the content is code.
I'd like to propose to allow XARs to be built form source code which is true source that one edits. Code files with their proper syntax coloring, editors, auto-completion, search. Media-files that are edited with our tool of choice by a simple double-click and not a complex pull, edit, and replace.
Thus, my proposal is to allow inclusions within the XML files. Inclusions should be of type: - text to include page content or object properties (e.g. css of cssx objects) - attachments to include attached objects
My current setup is documented here:http://jira.xwiki.org/browse/XWIKI-13643 It does binaryInclude to generate base64 and textInclude to generate XML-text. Thus, I can advertise a github project which is a macro and whose complex function is in a velocity file that is in a .vm file. However, I do not need to explode everything, e.g. the text-content describing the macro or (in this case too small) translation set.
What do you think? Should I rather generate complete attachment elements (name, length, modif date, ...) instead of doing a binaryInclude that just generates base64 ? Is fetching files for inclusion beyond the boundaries of the project of any use? (it is currently prohibited for safety)
thanks in advance.
Paul
participants (4)
-
Guillaume Delhumeau -
Paul Libbrecht -
Thomas Mortagne -
Vincent Massol