Re: [xwiki-devs] [proposal] Move away from XWQL, it is invalid JPQL (JPA) thus it is it's own (undefined) standard.
6 Apr
2011
6 Apr
'11
3:54 a.m.
Le 04/04/11 19:58, Caleb James DeLisle a écrit :
>
> On 04/04/2011 12:47 PM, Ludovic Dubost wrote:
>> Le 04/04/11 16:37, Caleb James DeLisle a écrit :
>>> On 04/04/2011 10:01 AM, Sergiu Dumitriu wrote:
>>>> On 04/02/2011 02:22 PM, Caleb James DeLisle wrote:
>>>>> After searching through documentation on JPQL (JPA's query language) I was unable to find any
>>>>> example of the "doc.object(XWiki.XWikiUsers)" construct. This means XWQL is it's own standard and
>>>>> there is no authoritative reference on it. What makes an implementation compliant? I have found
>>>>> that
>>>>> most HQL queries can be executed as XWQL queries with little or no modification so if compliance is
>>>>> defined as being "just like the reference implementation" then nearly all HQL must be
>>>>> implemented in
>>>>> order to be compliant.
>>>> The goal of XWQL was to not be bound to a certain query language, but to
>>>> be able to map it to as many QLs as possible, be they SQL-related, like
>>>> HQL or JPQL, or other types of queries, like QBE, XPath, SPARQL. So, it
>>>> wasn't meant from the start to be compatible with any standard.
>>> The problem now is we don't have any specification to tell us what is valid and what is not.
>>> Is this a valid XWQL query?
>>>
>>> $services.query.xwql("from BaseObject as obj where doc.fullName = obj.name and obj.className =
>>> 'XWiki.XWikiUsers'").execute()
>>>
>>> Run it and you might be surprised.
>>> Based on that, we have no way of ensuring that a query which works now will work in a new XWQL
>>> implementation which defeats the purpose of abstracting the user away from HQL.
>>>
>>>> Now, I'm not sure if the right thing to do is to move to a standard
>>>> query language, or to stick with our own.
>>> If we're going to define our own query language (I think there are enough already) there are certain
>>> things we have to do such as writing a specification. I frankly find this thing embarrassing.
>>>
>>>> - Is there any tool that allows mapping a JPQL or JDOQL query into other
>>>> query languages?
>>> http://www.datanucleus.org/products/accessplatform_3_0/datastores.html
>>> These folks are mapping JDOQL and JPQL into a whole bunch of different types of storage.
>>>
>>>> - Is there a way to parse a query into a tree/AST?
>>>> - Other than the fact that it's a non-standard language (and all the
>>>> consequences of this, like no support from tools and libraries), are
>>>> there any downsides to having our own query language?
>>> This particular one has 2 downsides:
>>> 1. There is no official specification.
>>> 2. HQL can be run as shown above.
>>>
>>> The major downside of implementing one correctly is that it is massively complicated.
>>>
>>> Caleb
>>>
>>>> The benefit of XWQL was that it allowed to write domain specific
>>>> queries, which are shorter and easier to understand (at least in theory).
>>>>
>>>>> Looking at the specifications I have rewritten the example query in compliant JPQL and JDOQL.
>>>>> I wrote these so that they would work if all objects were custom mapped which is similar to the
>>>>> appearance XWQL gives.
>>>>>
>>>>> XWQL:
>>>>> (SELECT doc.fullName FROM XWikiDocument as doc) where doc.author = 'XWiki.LudovicDubost' and
>>>>> doc.object(XWiki.XWikiUsers).email like '%xwiki.com'
>>>>>
>>>>> JPQL:
>>>>> SELECT doc.fullName FROM XWikiDocument as doc, IN(doc.xObjects) obj WHERE obj.className =
>>>>> 'XWiki.XWikiUsers' and obj.email LIKE '%xwiki.com'
>>>>>
>>>>> JDOQL:
>>>>> SELECT this.fullName FROM XWikiDocument WHERE this.xObjects.containsValue(obj)&& obj.className ==
>>>>> "XWiki.XWikiUsers"&& obj.email.startsWith("xwiki.com")
>>>>>
>> The key objective of XWQL is to abstract from the XWiki point of view and make it as simple as
>> possible to write queries.
>> If I take this (valid) query in XWQL:
>>
>> from doc.object(Blog.BlogPostClass) as blogarticle where 'Blog.Blogging' member of blogarticle.category
> I'm not 100% sure on this since I don't have a JDOQL database here but I think this will work:
> (assuming we still start the query with "SELECT doc.fullName from XWikiDocument as doc where ")
>
> doc.xObjects.containsValue(article)&& article.className == "Blog.Blogging"&&
> article.category.contains("Blog.Blogging")
>
> It is 18 letters longer, does 18 letters justify throwing out a perfectly good (and well thought
> out) specification and countless pages of tutorials and reference material?
Well we REALLY need to make this simple for users. The main complaint in
HQL is the additional useless joins.
Also we want as natural ways to write things. Users are used to SQL.
In your example there is:
doc.xObjects.containsValue(article)&& article.className == "Blog.Blogging"
instead of
from doc.object(Blog.BlogPostClass) as blogarticle
Franckly the "doc.xObjects.containsValue(article)" sounds very weird.
This would yet again be something users won't be able to master.
We want to be as close to:
"from Blog.BlogPostClass as article where article.title like '%xxx%'"
I think XWQL is very close from that.
Ludovic
>> It seems to me that it would be more complex in JPQL or JDOQL, which is not very cool.
>> I'm -1 from any new query language that would end up being more complex.
>>
>> If I take your 2 downsides:
>>
>> 1/ Official spec
>>
>> Well we should write one. It seems normal to me that a new query language has no spec.
>> But this does not mean that we don't need a language.
> Usually things go the other way, if you write an implementation then define the spec from it, how do
> you know a bug from a feature?
>
>> Unless we can find a standard language that is as easy to use as XWQL, let's stick to XWQL.
>> If we need to improve XWQL let's improve it.
> How much effort are willing to invest? A query language is like a small programming language. It is
> not something to be taken lightly.
>
>> 2/ HQL can be run
>>
>> That's an implementation issue. XWQL's objective is to write and run after translations to whatever
>> is needed by the backend.
> Since we are lacking any specification, the implementation is all we have. To write a spec we will
> have to ask "is that a bug or a feature?" with every possible construct.
>
>> It is VERY important to have a simple query language. It took a long time to learn how to run joins
>> in HQL which are not needed from a pure "functionnal" standpoint.
> Total +1 here, nobody should ever have to say JOIN in a query. Fortunately JOIN is not even a
> keyword in JDOQL :)
>
> Caleb
>
>> BTW I had fixed XWiki's query generator which now generates XWQL. It needs XWiki 2.7.1+
>>
>> Example here:
>>
>> http://www.ludovic.org/xwiki/bin/view/Test/Query?query=1&classname=Blog.BlogPostClass&Blog.BlogPostClass_title=&Blog.BlogPostClass_content=&Blog.BlogPostClass_extract=&Blog.BlogPostClass_category=Blog.Blogging&Blog.BlogPostClass_publishDate_morethan=&Blog.BlogPostClass_publishDate_lessthan=
>>
>>
>> I will document it and publish it on extensions.xwiki.org
>>
>> Ludovic
>>
>>
>>>>> I understood that XWQL was simply a translation scheme which made it appear that we were using JPQL
>>>>> with the schema we wanted when really we were using HQL with the schema we had. Given that it is
>>>>> not
>>>>> compliant JPQL that is not the case.
>>>>>
>>>>> I think when we update the schema, we should cut our losses with this thing and move to something
>>>>> which has a reference document and is more widely used.
>>>>>
>>>>> WDYT?
>>>>>
>>>>> Caleb
>>>>>
>>>>>
>>>>>
>>>>> The JPQL specification (originally called EJBQL):
>>>>> ejb-3_0-fr-spec-ejbcore.pdf chapter 9.
>>>>> http://jcp.org/aboutJava/communityprocess/final/jsr220/
>>>>>
>>>>> The JDOQL specification:
>>>>> jdo-3_0-mrel3-spec.pdf chapter 26.
>>>>> http://db.apache.org/jdo/specifications.html
>>>>>
>>>>> Easy to read, example rich descriptions:
>>>>> http://www.datanucleus.org/products/accessplatform_3_0/jpa/jpql.html
>>>>> http://www.datanucleus.org/products/accessplatform_3_0/jdo/jdoql.html
>>> _______________________________________________
>>> devs mailing list
>>> devs@xwiki.org
>>> http://lists.xwiki.org/mailman/listinfo/devs
>>>
>>
>
--
Ludovic Dubost
Blog: http://blog.ludovic.org/
XWiki: http://www.xwiki.com
Skype: ldubost GTalk: ldubost
6 Apr
6 Apr
10:38 a.m.
New subject: [xwiki-devs] [proposal] Move away from XWQL, it is invalid JPQL (JPA) thus it is it's own (undefined) standard.
This message was meant for the list but I sent to Ludovic by mistake.
Caleb
On 04/05/2011 09:54 PM, Ludovic Dubost wrote:
> Le 04/04/11 19:58, Caleb James DeLisle a écrit :
>>
>> On 04/04/2011 12:47 PM, Ludovic Dubost wrote:
>>> Le 04/04/11 16:37, Caleb James DeLisle a écrit :
>>>> On 04/04/2011 10:01 AM, Sergiu Dumitriu wrote:
>>>>> On 04/02/2011 02:22 PM, Caleb James DeLisle wrote:
>>>>>> After searching through documentation on JPQL (JPA's query language) I was unable to find any
>>>>>> example of the "doc.object(XWiki.XWikiUsers)" construct. This means XWQL is it's own standard and
>>>>>> there is no authoritative reference on it. What makes an implementation compliant? I have found
>>>>>> that
>>>>>> most HQL queries can be executed as XWQL queries with little or no modification so if
>>>>>> compliance is
>>>>>> defined as being "just like the reference implementation" then nearly all HQL must be
>>>>>> implemented in
>>>>>> order to be compliant.
>>>>> The goal of XWQL was to not be bound to a certain query language, but to
>>>>> be able to map it to as many QLs as possible, be they SQL-related, like
>>>>> HQL or JPQL, or other types of queries, like QBE, XPath, SPARQL. So, it
>>>>> wasn't meant from the start to be compatible with any standard.
>>>> The problem now is we don't have any specification to tell us what is valid and what is not.
>>>> Is this a valid XWQL query?
>>>>
>>>> $services.query.xwql("from BaseObject as obj where doc.fullName = obj.name and obj.className =
>>>> 'XWiki.XWikiUsers'").execute()
>>>>
>>>> Run it and you might be surprised.
>>>> Based on that, we have no way of ensuring that a query which works now will work in a new XWQL
>>>> implementation which defeats the purpose of abstracting the user away from HQL.
>>>>
>>>>> Now, I'm not sure if the right thing to do is to move to a standard
>>>>> query language, or to stick with our own.
>>>> If we're going to define our own query language (I think there are enough already) there are
>>>> certain
>>>> things we have to do such as writing a specification. I frankly find this thing embarrassing.
>>>>
>>>>> - Is there any tool that allows mapping a JPQL or JDOQL query into other
>>>>> query languages?
>>>> http://www.datanucleus.org/products/accessplatform_3_0/datastores.html
>>>> These folks are mapping JDOQL and JPQL into a whole bunch of different types of storage.
>>>>
>>>>> - Is there a way to parse a query into a tree/AST?
>>>>> - Other than the fact that it's a non-standard language (and all the
>>>>> consequences of this, like no support from tools and libraries), are
>>>>> there any downsides to having our own query language?
>>>> This particular one has 2 downsides:
>>>> 1. There is no official specification.
>>>> 2. HQL can be run as shown above.
>>>>
>>>> The major downside of implementing one correctly is that it is massively complicated.
>>>>
>>>> Caleb
>>>>
>>>>> The benefit of XWQL was that it allowed to write domain specific
>>>>> queries, which are shorter and easier to understand (at least in theory).
>>>>>
>>>>>> Looking at the specifications I have rewritten the example query in compliant JPQL and JDOQL.
>>>>>> I wrote these so that they would work if all objects were custom mapped which is similar to the
>>>>>> appearance XWQL gives.
>>>>>>
>>>>>> XWQL:
>>>>>> (SELECT doc.fullName FROM XWikiDocument as doc) where doc.author = 'XWiki.LudovicDubost' and
>>>>>> doc.object(XWiki.XWikiUsers).email like '%xwiki.com'
>>>>>>
>>>>>> JPQL:
>>>>>> SELECT doc.fullName FROM XWikiDocument as doc, IN(doc.xObjects) obj WHERE obj.className =
>>>>>> 'XWiki.XWikiUsers' and obj.email LIKE '%xwiki.com'
>>>>>>
>>>>>> JDOQL:
>>>>>> SELECT this.fullName FROM XWikiDocument WHERE this.xObjects.containsValue(obj)&&
>>>>>> obj.className ==
>>>>>> "XWiki.XWikiUsers"&& obj.email.startsWith("xwiki.com")
>>>>>>
>>> The key objective of XWQL is to abstract from the XWiki point of view and make it as simple as
>>> possible to write queries.
>>> If I take this (valid) query in XWQL:
>>>
>>> from doc.object(Blog.BlogPostClass) as blogarticle where 'Blog.Blogging' member of
>>> blogarticle.category
>> I'm not 100% sure on this since I don't have a JDOQL database here but I think this will work:
>> (assuming we still start the query with "SELECT doc.fullName from XWikiDocument as doc where ")
>>
>> doc.xObjects.containsValue(article)&& article.className == "Blog.Blogging"&&
>> article.category.contains("Blog.Blogging")
>>
>> It is 18 letters longer, does 18 letters justify throwing out a perfectly good (and well thought
>> out) specification and countless pages of tutorials and reference material?
>
> Well we REALLY need to make this simple for users. The main complaint in HQL is the additional
> useless joins.
> Also we want as natural ways to write things. Users are used to SQL.
>
> In your example there is:
>
> doc.xObjects.containsValue(article)&& article.className == "Blog.Blogging"
>
> instead of
>
> from doc.object(Blog.BlogPostClass) as blogarticle
>
>
> Franckly the "doc.xObjects.containsValue(article)" sounds very weird.
> This would yet again be something users won't be able to master.
That's JDOQL which is nice IMO but it is strange for someone who knows SQL.
With the type of schema I imagine, a JPQL query would look like:
, IN(doc.xObjects) blogarticle WHERE blogarticle.className = 'Blog.BlogPostClass' and
'Blog.Blogging' MEMBER OF blogarticle.category
For reference the HQL using our existing schema would look like:
, ListObject as obj, ListProperty as prop WHERE prop.name = 'category' AND prop.value =
'Blog.Blogging' AND prop.id = obj.id AND obj.name = doc.fullName AND obj.className =
'Blog.BlogPostClass'
So if we look at JPQL, JDOQL, HQL (with the current schema) and XWQL, we have:
JPQL - Reasonably concise, mostly like SQL.
JDOQL - Very concise, hard for SQL people to understand but easy for people who know the XWiki API
since the commands are the same as API method calls.
XWQL - Very concise, nonstandard, not really defined or documented.
HQL (with existing schema) - extremely verbose, documented but nonstandard (it's just something
someone at RedHat invented.)
This all gives me an idea. Maintaining a query language is a very big task but if we really need
that added simplicity of calling doc.objects(BlogClass) then why not specify an extension to JPQL?
By specifying an extension, the specification job, the documentation job, the testing job and the
implementation job are all brought down to reasonable levels. It is also something that we can and
should propose to the java community since a concise way to get a field from a map is something
missing from JPQL.
The allowed ways to get a field from a map in JPQL are the above and the following:
, JOIN doc.xObjects entry WHERE KEY(entry) = 'Blog.BlogPostClass' and 'Blog.Blogging' MEMBER OF
VALUE(entry).category
There is clearly room for a more concise method and although I still would prefer standards
compliance, I would much prefer defining an extension to a well defined query language than going it
alone. This would alleviate 95% of my concern.
Caleb
>
> We want to be as close to:
>
> "from Blog.BlogPostClass as article where article.title like '%xxx%'"
>
> I think XWQL is very close from that.
>
> Ludovic
>>> It seems to me that it would be more complex in JPQL or JDOQL, which is not very cool.
>>> I'm -1 from any new query language that would end up being more complex.
>>>
>>> If I take your 2 downsides:
>>>
>>> 1/ Official spec
>>>
>>> Well we should write one. It seems normal to me that a new query language has no spec.
>>> But this does not mean that we don't need a language.
>> Usually things go the other way, if you write an implementation then define the spec from it, how do
>> you know a bug from a feature?
>>
>>> Unless we can find a standard language that is as easy to use as XWQL, let's stick to XWQL.
>>> If we need to improve XWQL let's improve it.
>> How much effort are willing to invest? A query language is like a small programming language. It is
>> not something to be taken lightly.
>>
>>> 2/ HQL can be run
>>>
>>> That's an implementation issue. XWQL's objective is to write and run after translations to whatever
>>> is needed by the backend.
>> Since we are lacking any specification, the implementation is all we have. To write a spec we will
>> have to ask "is that a bug or a feature?" with every possible construct.
>>
>>> It is VERY important to have a simple query language. It took a long time to learn how to run joins
>>> in HQL which are not needed from a pure "functionnal" standpoint.
>> Total +1 here, nobody should ever have to say JOIN in a query. Fortunately JOIN is not even a
>> keyword in JDOQL :)
>>
>> Caleb
>>
>>> BTW I had fixed XWiki's query generator which now generates XWQL. It needs XWiki 2.7.1+
>>>
>>> Example here:
>>>
>>> http://www.ludovic.org/xwiki/bin/view/Test/Query?query=1&classname=Blog.BlogPostClass&Blog.BlogPostClass_title=&Blog.BlogPostClass_content=&Blog.BlogPostClass_extract=&Blog.BlogPostClass_category=Blog.Blogging&Blog.BlogPostClass_publishDate_morethan=&Blog.BlogPostClass_publishDate_lessthan=
>>>
>>>
>>>
>>> I will document it and publish it on extensions.xwiki.org
>>>
>>> Ludovic
>>>
>>>
>>>>>> I understood that XWQL was simply a translation scheme which made it appear that we were using
>>>>>> JPQL
>>>>>> with the schema we wanted when really we were using HQL with the schema we had. Given that it is
>>>>>> not
>>>>>> compliant JPQL that is not the case.
>>>>>>
>>>>>> I think when we update the schema, we should cut our losses with this thing and move to something
>>>>>> which has a reference document and is more widely used.
>>>>>>
>>>>>> WDYT?
>>>>>>
>>>>>> Caleb
>>>>>>
>>>>>>
>>>>>>
>>>>>> The JPQL specification (originally called EJBQL):
>>>>>> ejb-3_0-fr-spec-ejbcore.pdf chapter 9.
>>>>>> http://jcp.org/aboutJava/communityprocess/final/jsr220/
>>>>>>
>>>>>> The JDOQL specification:
>>>>>> jdo-3_0-mrel3-spec.pdf chapter 26.
>>>>>> http://db.apache.org/jdo/specifications.html
>>>>>>
>>>>>> Easy to read, example rich descriptions:
>>>>>> http://www.datanucleus.org/products/accessplatform_3_0/jpa/jpql.html
>>>>>> http://www.datanucleus.org/products/accessplatform_3_0/jdo/jdoql.html
>>>> _______________________________________________
>>>> devs mailing list
>>>> devs@xwiki.org
>>>> http://lists.xwiki.org/mailman/listinfo/devs
>>>>
>>>
>>
>
>
5633
Age (days ago)
5633
Last active (days ago)
1 comments
2 participants
participants (2)
-
Caleb James DeLisle -
Ludovic Dubost