Re: [xwiki-users] Xwiki file and attachment storage
Esbach, Brandon wrote:
With a little work, you could probably intercept the upload via groovy, place it on a network drive, replace the attachment with a link to that network file.. <shrug> Would think that this would be your next step? Granted, I've never tested the idea; but it should do what you're looking for.
So I take it nothing like this is in the works? :) -- View this message in context: http://www.nabble.com/Xwiki-file-and-attachment-storage-tf4214945.html#a1238... Sent from the XWiki- Users mailing list archive at Nabble.com. _______________________________________________ users mailing list users@xwiki.org http://lists.xwiki.org/mailman/listinfo/users
So I take it nothing like this is in the works? :)
Actually, there are some ideas of providing WebDav access to an XWiki instance, but that's still using a database, but you can edit the files as they were on a real filesystem. Also, the JCR storage backend is almost functional, it just needs a little refreshing and cleanup. Sergiu -- http://purl.org/net/sergiu _______________________________________________ users mailing list users@xwiki.org http://lists.xwiki.org/mailman/listinfo/users
On Aug 29, 2007, at 5:46 PM, bjquinn wrote:
Esbach, Brandon wrote:
With a little work, you could probably intercept the upload via groovy, place it on a network drive, replace the attachment with a link to that network file.. <shrug> Would think that this would be your next step? Granted, I've never tested the idea; but it should do what you're looking for.
So I take it nothing like this is in the works? :)
like what? (did you see my answer btw) Thanks -Vincent _______________________________________________ users mailing list users@xwiki.org http://lists.xwiki.org/mailman/listinfo/users
vmassol wrote:
Artem, would you mind giving us a status on the JCR work and the plan ahead?
vmassol wrote:
So I take it nothing like this is in the works? :)
like what? (did you see my answer btw)
Just the part about storing attachments as files outside of the DB. Did Artem ever respond about the status of the JCR work? Would it in and of itself allow storing attachments as files outside of the DB or is it just a framework that would allow something like that if more work was done? -- View this message in context: http://www.nabble.com/Xwiki-file-and-attachment-storage-tf4214945.html#a1239... Sent from the XWiki- Users mailing list archive at Nabble.com. _______________________________________________ users mailing list users@xwiki.org http://lists.xwiki.org/mailman/listinfo/users
On Aug 30, 2007, at 1:51 AM, bjquinn wrote:
vmassol wrote:
Artem, would you mind giving us a status on the JCR work and the plan ahead?
vmassol wrote:
So I take it nothing like this is in the works? :)
like what? (did you see my answer btw)
Just the part about storing attachments as files outside of the DB.
Did Artem ever respond about the status of the JCR work? Would it in and of itself allow storing attachments as files outside of the DB or is it just a framework that would allow something like that if more work was done?
Google is your friend... :) For example, at random: http://www.onjava.com/pub/a/onjava/2006/10/04/what-is-java-content- repository.html -Vincent _______________________________________________ users mailing list users@xwiki.org http://lists.xwiki.org/mailman/listinfo/users
vmassol wrote:
Google is your friend... :)
For example, at random: http://www.onjava.com/pub/a/onjava/2006/10/04/what-is-java-content- repository.html
-Vincent
Sure, Google is my friend. :) What I was wanting to know, though, is whether the work you're doing will implement JCR in the sense that someone *could* write some code to allow attachments to be stored on the filesystem, or is allowing attachments to be stored on the filesystem *part* of the work that you're doing with JCR right now for a future xwiki release? Thanks! -BJ -- View this message in context: http://www.nabble.com/Xwiki-file-and-attachment-storage-tf4214945.html#a1240... Sent from the XWiki- Users mailing list archive at Nabble.com. _______________________________________________ users mailing list users@xwiki.org http://lists.xwiki.org/mailman/listinfo/users
On Aug 30, 2007, at 4:44 PM, bjquinn wrote:
vmassol wrote:
Google is your friend... :)
For example, at random: http://www.onjava.com/pub/a/onjava/2006/10/04/what-is-java-content- repository.html
-Vincent
Sure, Google is my friend. :)
What I was wanting to know, though, is whether the work you're doing will implement JCR in the sense that someone *could* write some code to allow attachments to be stored on the filesystem, or is allowing attachments to be stored on the filesystem *part* of the work that you're doing with JCR right now for a future xwiki release?
We simply use an existing JCR implementation and do the mapping between XWiki's internal Storage interface to it. The heavy lifting and third party support is done by the JCR implementation. I think the one we're using right now is Jackrabbit. So whatever Jackrabbit supports we support. Thanks -Vincent _______________________________________________ users mailing list users@xwiki.org http://lists.xwiki.org/mailman/listinfo/users
http://jackrabbit.apache.org/faq.html#whats-fs wrote:
What is a Jackrabbit file system?
A Jackrabbbit file system (FS) is an internal component that implements standard file system operations on top of some underlying storage mechanism (a normal file system, a database, a webdav server, or a custom file format). A file system component is any Java class that implements the FileSystem interface and the associated behavioural contracts. File systems are used in Jackrabbit both as subcomponents of the persistence managers and for general storage needs (for example to store the full text indexes).
So does this mean that I will be able to mount the xwiki database as a filesystem or something? Is that what you mean? And is this Jackrabbit stuff ready by v1.1? (Please excuse my ignorance! I'm a bit weak when it comes to Java!!!) -BJ -- View this message in context: http://www.nabble.com/Xwiki-file-and-attachment-storage-tf4214945.html#a1240... Sent from the XWiki- Users mailing list archive at Nabble.com. _______________________________________________ users mailing list users@xwiki.org http://lists.xwiki.org/mailman/listinfo/users
On Aug 30, 2007, at 5:19 PM, bjquinn wrote:
http://jackrabbit.apache.org/faq.html#whats-fs wrote:
What is a Jackrabbit file system?
A Jackrabbbit file system (FS) is an internal component that implements standard file system operations on top of some underlying storage mechanism (a normal file system, a database, a webdav server, or a custom file format). A file system component is any Java class that implements the FileSystem interface and the associated behavioural contracts. File systems are used in Jackrabbit both as subcomponents of the persistence managers and for general storage needs (for example to store the full text indexes).
So does this mean that I will be able to mount the xwiki database as a filesystem or something?
There won't be a xwiki database any more. All the data will be stored through the JCR implementation we use. It'll be up to you to decide what you want the JCR implementation to point to through configuration. I mentioned Jackrabbit because this is what our proof of concept uses but we might use eXo Platform, JackRabbit or some other implementation. XWiki will use the JCR API so it can use any JCR implementation (theoretically at least).
Is that what you mean? And is this Jackrabbit stuff ready by v1.1?
Nope. This is 1.2 stuff. -Vincent _______________________________________________ users mailing list users@xwiki.org http://lists.xwiki.org/mailman/listinfo/users
Is that what you mean? And is this Jackrabbit stuff ready by v1.1?
Nope. This is 1.2 stuff.
Well, 1.2 is almost here... is the jackrabbit stuff ready, and can I store attachments in files?? -- View this message in context: http://www.nabble.com/Xwiki-file-and-attachment-storage-tf4214945.html#a1349... Sent from the XWiki- Users mailing list archive at Nabble.com.
Hi, On Oct 30, 2007, at 9:03 PM, bjquinn wrote:
Is that what you mean? And is this Jackrabbit stuff ready by v1.1?
Nope. This is 1.2 stuff.
Well, 1.2 is almost here... is the jackrabbit stuff ready, and can I store attachments in files??
Sorry but no. The scope for 1.2 was changed a long time ago. More importantly Artem who was working on it has stopped participating to XWiki for some time since he's busy on other stuff. He'll join us back later but there's no ETA right now on the JCR implementation. However I'm curious to know why you need attachments stored in the file system. Thanks -Vincent
Vincent Massol wrote:
Sorry but no. The scope for 1.2 was changed a long time ago. More importantly Artem who was working on it has stopped participating to XWiki for some time since he's busy on other stuff. He'll join us back later but there's no ETA right now on the JCR implementation.
However I'm curious to know why you need attachments stored in the file system.
Thanks -Vincent Sorry if jumping in this thread is not the most elegant way of collaborate, but I've missed this option, to store attachments in the file system, today.
For instance: we use a lot of high resolution pictures in our work. If I can store this images in the file system I can access them by using "regular" drive mapping, web server access, sftp,... I mean: I could keep relying on different clients to read the files while I could rely on XWiki to easy the way users "load" files in the system. Hope this makes sense for you. Cheers, Ricardo -- Ricardo RodrÃguez Your XEN ICT Team
On Oct 31, 2007, at 12:01 AM, Your XEN ICT Team - Ricardo Rodriguez wrote:
Vincent Massol wrote:
Sorry but no. The scope for 1.2 was changed a long time ago. More =
importantly Artem who was working on it has stopped participating to =
XWiki for some time since he's busy on other stuff. He'll join us back =
later but there's no ETA right now on the JCR implementation.
However I'm curious to know why you need attachments stored in the =
file system.
Thanks -Vincent Sorry if jumping in this thread is not the most elegant way of =
collaborate, but I've missed this option, to store attachments in the =
file system, today.
For instance: we use a lot of high resolution pictures in our work. If I =
can store this images in the file system I can access them by using =
"regular" drive mapping, web server access, sftp,... I mean: I could =
keep relying on different clients to read the files while I could rely =
on XWiki to easy the way users "load" files in the system.
Hope this makes sense for you.
Definitely. For this use case, webdav support would probably be the best. We have jira issues opened for this but nobody has implemented it yet AFAIK. I guess if the JCR implementation also supports Webdav that should work too. -Vincent
Vincent Massol wrote:
Definitely. For this use case, webdav support would probably be the best. We have jira issues opened for this but nobody has implemented it yet AFAIK. I guess if the JCR implementation also supports Webdav that should work too.
-Vincent
_______________________________________________ users mailing list users@xwiki.org http://lists.xwiki.org/mailman/listinfo/users
Thanks, Vincent. I'll search and keep track of these issues. Unfortunately I'm still far from being able to contribute to such an effort. But keep trying hard! Cheers, -- Ricardo RodrÃguez Your XEN ICT Team
vmassol wrote:
However I'm curious to know why you need attachments stored in the file system.
Because many of these attachments may be large (>50MB), and over time the database can grow to be unweildy. Currently that's our problem with our exchange server setup (people keep emailing these files back and forth as it stands now *shudder*). The upside of storing attachments in a filesystem is easier access from other clients (as the other poster suggested), less likelihood of corruption on large databases (our Exchange DB is 200+ GB right now), less likelihood of backup problems, copy-only-what's-changed backups, and quicker backup and restore time. -- View this message in context: http://www.nabble.com/Xwiki-file-and-attachment-storage-tf4214945.html#a1379... Sent from the XWiki- Users mailing list archive at Nabble.com.
One more upside is that with filesystem attachments you can easily use streams and never hold whole attachment in memory. This is really necessary for scalable deployment. And BTW there is no max_packet_size restriction as well. One can also do streaming with db BLOBs, but not via hibernate I believe. -----Original Message----- From: users-bounces@xwiki.org [mailto:users-bounces@xwiki.org] On Behalf Of bjquinn Sent: Friday, November 16, 2007 9:14 PM To: users@xwiki.org Subject: Re: [xwiki-users] Xwiki file and attachment storage vmassol wrote:
However I'm curious to know why you need attachments stored in the file system.
Because many of these attachments may be large (>50MB), and over time the database can grow to be unweildy. Currently that's our problem with our exchange server setup (people keep emailing these files back and forth as it stands now *shudder*). The upside of storing attachments in a filesystem is easier access from other clients (as the other poster suggested), less likelihood of corruption on large databases (our Exchange DB is 200+ GB right now), less likelihood of backup problems, copy-only-what's-changed backups, and quicker backup and restore time. -- View this message in context: http://www.nabble.com/Xwiki-file-and-attachment-storage-tf4214945.html#a1379 9175 Sent from the XWiki- Users mailing list archive at Nabble.com. _______________________________________________ users mailing list users@xwiki.org http://lists.xwiki.org/mailman/listinfo/users
Just to be clear. We're all in favor of having the ability to store attachments using webdav. It'll happen eventually. the only thing lacking right now is manpower. Thanks -Vincent On Nov 16, 2007, at 8:47 PM, Pavel wrote:
One more upside is that with filesystem attachments you can easily use streams and never hold whole attachment in memory. This is really necessary for scalable deployment.
And BTW there is no max_packet_size restriction as well.
One can also do streaming with db BLOBs, but not via hibernate I believe.
-----Original Message----- From: users-bounces@xwiki.org [mailto:users-bounces@xwiki.org] On Behalf Of bjquinn Sent: Friday, November 16, 2007 9:14 PM To: users@xwiki.org Subject: Re: [xwiki-users] Xwiki file and attachment storage
vmassol wrote:
However I'm curious to know why you need attachments stored in the file system.
Because many of these attachments may be large (>50MB), and over time the database can grow to be unweildy. Currently that's our problem with our exchange server setup (people keep emailing these files back and forth as it stands now *shudder*). The upside of storing attachments in a filesystem is easier access from other clients (as the other poster suggested), less likelihood of corruption on large databases (our Exchange DB is 200+ GB right now), less likelihood of backup problems, copy-only-what's- changed backups, and quicker backup and restore time. -- View this message in context: http://www.nabble.com/Xwiki-file-and-attachment-storage-tf4214945.html#a1379 9175 Sent from the XWiki- Users mailing list archive at Nabble.com.
_______________________________________________ users mailing list users@xwiki.org http://lists.xwiki.org/mailman/listinfo/users
_______________________________________________ users mailing list users@xwiki.org http://lists.xwiki.org/mailman/listinfo/users
Well with v1.3 coming out and 1.4 on the horizon, I was just wondering if there had been any progress on this issue? Thanks! -BJ Quinn vmassol wrote:
Just to be clear. We're all in favor of having the ability to store attachments using webdav. It'll happen eventually.
the only thing lacking right now is manpower.
Thanks -Vincent
On Nov 16, 2007, at 8:47 PM, Pavel wrote:
One more upside is that with filesystem attachments you can easily use streams and never hold whole attachment in memory. This is really necessary for scalable deployment.
And BTW there is no max_packet_size restriction as well.
One can also do streaming with db BLOBs, but not via hibernate I believe.
-----Original Message----- From: users-bounces@xwiki.org [mailto:users-bounces@xwiki.org] On Behalf Of bjquinn Sent: Friday, November 16, 2007 9:14 PM To: users@xwiki.org Subject: Re: [xwiki-users] Xwiki file and attachment storage
vmassol wrote:
However I'm curious to know why you need attachments stored in the file system.
Because many of these attachments may be large (>50MB), and over time the database can grow to be unweildy. Currently that's our problem with our exchange server setup (people keep emailing these files back and forth as it stands now *shudder*). The upside of storing attachments in a filesystem is easier access from other clients (as the other poster suggested), less likelihood of corruption on large databases (our Exchange DB is 200+ GB right now), less likelihood of backup problems, copy-only-what's- changed backups, and quicker backup and restore time. -- View this message in context: http://www.nabble.com/Xwiki-file-and-attachment-storage-tf4214945.html#a1379 9175 Sent from the XWiki- Users mailing list archive at Nabble.com.
_______________________________________________ users mailing list users@xwiki.org http://lists.xwiki.org/mailman/listinfo/users
_______________________________________________ users mailing list users@xwiki.org http://lists.xwiki.org/mailman/listinfo/users
_______________________________________________ users mailing list users@xwiki.org http://lists.xwiki.org/mailman/listinfo/users
-- View this message in context: http://www.nabble.com/Xwiki-file-and-attachment-storage-tp11991466p16048620.... Sent from the XWiki- Users mailing list archive at Nabble.com.
Sergiu was writing about it recently : http://markmail.org/message/egls2u2anv6ttfs2?q=xwiki+list:org%2Exwiki%2Edevs... BTW I don't know if there's any ETA for this, Sergiu have you planned to work on this in the coming weeks ? JV. On Fri, Mar 14, 2008 at 6:16 PM, bjquinn <bjquinn@seidal.com> wrote:
Well with v1.3 coming out and 1.4 on the horizon, I was just wondering if there had been any progress on this issue?
Thanks!
-BJ Quinn
vmassol wrote:
Just to be clear. We're all in favor of having the ability to store attachments using webdav. It'll happen eventually.
the only thing lacking right now is manpower.
Thanks -Vincent
On Nov 16, 2007, at 8:47 PM, Pavel wrote:
One more upside is that with filesystem attachments you can easily use streams and never hold whole attachment in memory. This is really necessary for scalable deployment.
And BTW there is no max_packet_size restriction as well.
One can also do streaming with db BLOBs, but not via hibernate I believe.
-----Original Message----- From: users-bounces@xwiki.org [mailto:users-bounces@xwiki.org] On Behalf Of bjquinn Sent: Friday, November 16, 2007 9:14 PM To: users@xwiki.org Subject: Re: [xwiki-users] Xwiki file and attachment storage
vmassol wrote:
However I'm curious to know why you need attachments stored in the file system.
Because many of these attachments may be large (>50MB), and over time the database can grow to be unweildy. Currently that's our problem with our exchange server setup (people keep emailing these files back and forth as it stands now *shudder*). The upside of storing attachments in a filesystem is easier access from other clients (as the other poster suggested), less likelihood of corruption on large databases (our Exchange DB is 200+ GB right now), less likelihood of backup problems, copy-only-what's- changed backups, and quicker backup and restore time. -- View this message in context: http://www.nabble.com/Xwiki-file-and-attachment-storage-tf4214945.html#a1379 9175 Sent from the XWiki- Users mailing list archive at Nabble.com.
_______________________________________________ users mailing list users@xwiki.org http://lists.xwiki.org/mailman/listinfo/users
_______________________________________________ users mailing list users@xwiki.org http://lists.xwiki.org/mailman/listinfo/users
_______________________________________________ users mailing list users@xwiki.org http://lists.xwiki.org/mailman/listinfo/users
-- View this message in context: http://www.nabble.com/Xwiki-file-and-attachment-storage-tp11991466p16048620....
Sent from the XWiki- Users mailing list archive at Nabble.com.
_______________________________________________ users mailing list users@xwiki.org http://lists.xwiki.org/mailman/listinfo/users
participants (6)
-
bjquinn -
Jean-Vincent Drean -
Pavel -
Sergiu Dumitriu -
Vincent Massol -
Your XEN ICT Team - Ricardo Rodriguez