[xwiki-users] Scripting - Saving calculated results for a macro
I am writing a macro that performs a relatively expensive calculation to generate content for display (a PNG image) based on user input. The basic macro works fine, but I'd like to cache the generated content to save recalculating it when I don't need to. My idea is that I generate a SHA1 digest of the macro content, and persistently cache the output generated for that input. I'm thinking of using an LRU cache to avoid keeping massive amounts of data that'll never get used again. The question I have is where I can put the cache so that it is saved with the page, and I can access and update it from the macro code. I suspect that the answer is to attach an object to the page somehow, but I've been looking round the documentation and can't find any examples of how I might do this (TBH, I'm struggling to find my way round the documentation, so it may be that I've missed something obvious - if so, then by all means point me to the right place :-)) As the object is a PNG image, I'm currently base64-encoding it, and using a data URL to display it. But if it's possible to cache the data in such a way that the raw data is (or can be made) directly addressible via a URL, that would be even better. Can anyone help? Thanks, Paul.
Paul Check the cache macro which you can use inside your own macro http://extensions.xwiki.org/xwiki/bin/view/Extension/Cache+Macro Envoyé de mon iPhone Le 26 mai 2012 à 14:52, Paul Moore <p.f.moore@gmail.com> a écrit :
I am writing a macro that performs a relatively expensive calculation to generate content for display (a PNG image) based on user input. The basic macro works fine, but I'd like to cache the generated content to save recalculating it when I don't need to.
My idea is that I generate a SHA1 digest of the macro content, and persistently cache the output generated for that input. I'm thinking of using an LRU cache to avoid keeping massive amounts of data that'll never get used again. The question I have is where I can put the cache so that it is saved with the page, and I can access and update it from the macro code. I suspect that the answer is to attach an object to the page somehow, but I've been looking round the documentation and can't find any examples of how I might do this (TBH, I'm struggling to find my way round the documentation, so it may be that I've missed something obvious - if so, then by all means point me to the right place :-))
As the object is a PNG image, I'm currently base64-encoding it, and using a data URL to display it. But if it's possible to cache the data in such a way that the raw data is (or can be made) directly addressible via a URL, that would be even better.
Can anyone help?
Thanks, Paul. _______________________________________________ users mailing list users@xwiki.org http://lists.xwiki.org/mailman/listinfo/users
On 26 May 2012 14:02, Ludovic Dubost <ludovic@xwiki.com> wrote:
Paul
Check the cache macro which you can use inside your own macro
http://extensions.xwiki.org/xwiki/bin/view/Extension/Cache+Macro
Interesting, thanks. I'm not quite sure how it would work in practice, it seems that I'd end up with macros nested up to 3 times, at which point my brain explodes :-) It might help if I show the content of my current macro here: {{groovy}} import java.security.* // Calculate SHA1 of content - the idea is that we can avoid regenerating if we've calculated this output before def content = xcontext.macro.content def messageDigest = MessageDigest.getInstance("SHA1") messageDigest.update(content.getBytes()) def contentSHA1 = new BigInteger(1, messageDigest.digest()).toString(16).padLeft(40, '0') // Generate the output - ideally, only if we don't have saved output for this SHA1 def cmd = 'generate-png.exe' def proc = cmd.execute() proc << content proc.out.close() def png = proc.in.bytes def out = png.encodeBase64().toString() println "{{html}}" println "<img src=\"data:image/png;base64,$out\"/>" println "{{/html}}" {{/groovy}} It's running that executable that I want to skip - it will always give the same output for the same input. The output should be persistent (over restarts of XWiki, ideally). As a concrete example of generate-png.exe, think of the Graphviz "dot" command. For any given graph, the PNG won't change. Essentially, that's what I'm implementing here (longer term, the idea would be that the precise command to generate the image would be configurable). I've looked at the PlantUML and Graphviz extensions, but I don't really follow how they work. On the other hand, they seem pretty complex compared to the code above, so I'm not sure how much understanding them would help. Paul.
On 05/26/2012 08:52 AM, Paul Moore wrote:
I am writing a macro that performs a relatively expensive calculation to generate content for display (a PNG image) based on user input. The basic macro works fine, but I'd like to cache the generated content to save recalculating it when I don't need to.
My idea is that I generate a SHA1 digest of the macro content, and persistently cache the output generated for that input. I'm thinking of using an LRU cache to avoid keeping massive amounts of data that'll never get used again. The question I have is where I can put the cache so that it is saved with the page, and I can access and update it from the macro code. I suspect that the answer is to attach an object to the page somehow, but I've been looking round the documentation and can't find any examples of how I might do this (TBH, I'm struggling to find my way round the documentation, so it may be that I've missed something obvious - if so, then by all means point me to the right place :-))
As the object is a PNG image, I'm currently base64-encoding it, and using a data URL to display it. But if it's possible to cache the data in such a way that the raw data is (or can be made) directly addressible via a URL, that would be even better.
Can anyone help?
Thanks, Paul.
Some resources that you can use: https://github.com/xwiki/xwiki-platform/blob/master/xwiki-platform-core/xwik... is the proper way of creating caches. You declare an @Inject dependency on that, then use the createNewCache method (if you don't plan on running a cluster, or don't think that the cached resources should be shared inside a cluster, then it's slightly better for performance to call createNewLocalCache). You can then use this cache to store some entries. https://github.com/xwiki/xwiki-commons/blob/master/xwiki-commons-core/xwiki-... is going to give you the location of the temporary storage directory. Combine this with https://github.com/xwiki/xwiki-platform/blob/master/xwiki-platform-core/xwik... and you have a nice way of serving image files stored on the disk to the HTTP client. Unfortunately, this way you have to implement your own LRU functionality and make sure you clean up files, otherwise you'll keep filling up the disk with temporary files. -- Sergiu Dumitriu http://purl.org/net/sergiu/
On 27 May 2012 04:11, Sergiu Dumitriu <sergiu@xwiki.com> wrote:
Some resources that you can use:
https://github.com/xwiki/xwiki-platform/blob/master/xwiki-platform-core/xwik... is the proper way of creating caches. You declare an @Inject dependency on that, then use the createNewCache method (if you don't plan on running a cluster, or don't think that the cached resources should be shared inside a cluster, then it's slightly better for performance to call createNewLocalCache). You can then use this cache to store some entries.
https://github.com/xwiki/xwiki-commons/blob/master/xwiki-commons-core/xwiki-... is going to give you the location of the temporary storage directory. Combine this with https://github.com/xwiki/xwiki-platform/blob/master/xwiki-platform-core/xwik... and you have a nice way of serving image files stored on the disk to the HTTP client. Unfortunately, this way you have to implement your own LRU functionality and make sure you clean up files, otherwise you'll keep filling up the disk with temporary files.
That's brilliant - thanks for those, I'll go and do some reading now. Paul.
On 27 May 2012 08:28, Paul Moore <p.f.moore@gmail.com> wrote:
https://github.com/xwiki/xwiki-platform/blob/master/xwiki-platform-core/xwik... and you have a nice way of serving image files stored on the disk to the HTTP client. Unfortunately, this way you have to implement your own LRU functionality and make sure you clean up files, otherwise you'll keep filling up the disk with temporary files.
That's brilliant - thanks for those, I'll go and do some reading now.
One quick question. I gather from what I read that this uses the "old way" of creating actions which involves registering them in the struts xml file. (I hope I got that right, I don't really understand all of this stuff very well). I saw a few mentions of the possibility of new-style components being able to register custom actions - but I got the impression that this was a plan rather than a reality. One of the problems I've found with looking for information about XWiki on the web is that a lot of things are somewhat out of date - so my question, is there a new-style way of creating a component that includes an action similar to the TempResourceAction code here? But that's just to save having to write messy tempfile-maintenance code, and is really very much a "nice to have". Thanks, Paul.
participants (3)
-
Ludovic Dubost -
Paul Moore -
Sergiu Dumitriu