From wshager at gmail.com  Thu Jan 12 04:21:25 2017
From: wshager at gmail.com (W.S. Hager)
Date: Thu, 12 Jan 2017 13:21:25 +0100
Subject: [xquery-talk] Fold-right
Message-ID: <CAP8fv=0gkmcc0-iYfyqvfW632kdOa8er4M+i+YKX5gZjquunJw@mail.gmail.com>

Hello,

Not sure if anyone already noticed, but there's an error in the
XPath/XQuery F&O docs, where the example implementation of array:fold will
always return an array...

More importantly, it seems to me that a textbook implementation of
fold-right was used, without the notice that this will only work in a lazy
evaluation context. If you inspect the code as eagerly evaluated, you'll
see that the array is looped over multiple times (the result is the same of
course). This makes for a very inefficient loop AFAICT, but I may be
mistaken.

I just discovered that Clojure doesn't even have fold-right, because it
isn't lazily evaluated (since Clojure isn't interpreted it's probably too
complicated on the JVM).

I guess for XQuery an interpreter could defer evaluation when it encounters
specific calls, but a superficial web search on this subject related to
e.g. Saxon doesn't provide specific answers, only that (understandably) the
number of performance improvements has grown over time, and perhaps
deferring evaluation is one of them. However, as I'm re-evaluating (no pun
intended) my investment in XQuery, I'm eager to hear if there are some more
theoretical grounds for providing said example, or if the committee intends
to one day provide them.

Is there any assumption about the laziness of an XQuery implementation? Are
there any a priori lazy implementations out there?

Thanks,
Wouter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://x-query.com/pipermail/talk/attachments/20170112/78918c37/attachment.html>

From liam at w3.org  Thu Jan 12 05:02:54 2017
From: liam at w3.org (Liam R. E. Quin)
Date: Thu, 12 Jan 2017 08:02:54 -0500
Subject: [xquery-talk] Fold-right
In-Reply-To: <CAP8fv=0gkmcc0-iYfyqvfW632kdOa8er4M+i+YKX5gZjquunJw@mail.gmail.com>
References: <CAP8fv=0gkmcc0-iYfyqvfW632kdOa8er4M+i+YKX5gZjquunJw@mail.gmail.com>
Message-ID: <1484226174.26891.113.camel@w3.org>

On Thu, 2017-01-12 at 13:21 +0100, W.S. Hager wrote:
> Hello,
> 
> Not sure if anyone already noticed, but there's an error in the
> XPath/XQuery F&O docs, where the example implementation of array:fold
> will always return an array...
I don't believe so. Please open a bug report (see the status section of
the document for details on how to do that).

> More importantly, it seems to me that a textbook implementation of
> fold-right was used, without the notice that this will only work in a
> lazy evaluation context.

XQuery is designed so that a hybrid lazy evaluation strategy is
encouraged. Queries are rewritten extensively by many implementations.
Having said that, we don't provide e.g. infinite sequences, so it's
also possible to implement an entirely eager XQuery.

The example implementations here and there in the spec are not
necessarily efficient but they should (unless otherwise noted in the
text) be correct in all cases.

Liam


-- 
Liam R. E. Quin <liam at w3.org>
The World Wide Web Consortium (W3C)

From mike at saxonica.com  Thu Jan 12 05:09:22 2017
From: mike at saxonica.com (Michael Kay)
Date: Thu, 12 Jan 2017 13:09:22 +0000
Subject: [xquery-talk] Fold-right
In-Reply-To: <CAP8fv=0gkmcc0-iYfyqvfW632kdOa8er4M+i+YKX5gZjquunJw@mail.gmail.com>
References: <CAP8fv=0gkmcc0-iYfyqvfW632kdOa8er4M+i+YKX5gZjquunJw@mail.gmail.com>
Message-ID: <698D3F1D-27D6-4FDB-9813-6414C11F650A@saxonica.com>


> On 12 Jan 2017, at 12:21, W.S. Hager <wshager at gmail.com> wrote:
> 
> Hello,
> 
> Not sure if anyone already noticed, but there's an error in the XPath/XQuery F&O docs, where the example implementation of array:fold will always return an array...

If they have noticed, they haven't reported it. Logged here as a bug:

https://www.w3.org/Bugs/Public/show_bug.cgi?id=30045

If you spot such things in future, please report them using the feedback mechanisms described in the status section of the spec.
> 
> More importantly,

No, I think the subsequent points are far less important. The code used to specify the effect of the function is intended to be a specification, not an efficient implementation.

> it seems to me that a textbook implementation of fold-right was used, without the notice that this will only work in a lazy evaluation context. If you inspect the code as eagerly evaluated, you'll see that the array is looped over multiple times (the result is the same of course). This makes for a very inefficient loop AFAICT, but I may be mistaken.

I don't think it's true that it only works with lazy evaluation. It may well be true that it only works efficiently with lazy evaluation, but that's irrelevant: this is a specfication of the effect of the function, not a proposed implementation.
> 
> 
> Is there any assumption about the laziness of an XQuery implementation? Are there any a priori lazy implementations out there?
> 

I think an implementation without lazy evaluation would be incredibly inefficient, and I would expect that most serious implementations use lazy evaluation to a lesser or greater extent. Certainly Saxon does so. I have seen open source XPath implementations that materialize all intermediate results, and frankly I don't think such an implementation is fit for purpose.

As it happens, this very morning I've been working on a problem where excessive use of lazy evaluation causes stack overflow problems when fold-left is used to process a very long sequence: you can find it here if you're interested:

https://saxonica.plan.io/issues/3106

Michael Kay
Saxonica



From wshager at gmail.com  Thu Jan 12 07:17:39 2017
From: wshager at gmail.com (W.S. Hager)
Date: Thu, 12 Jan 2017 16:17:39 +0100
Subject: [xquery-talk] Fold-right
In-Reply-To: <698D3F1D-27D6-4FDB-9813-6414C11F650A@saxonica.com>
References: <CAP8fv=0gkmcc0-iYfyqvfW632kdOa8er4M+i+YKX5gZjquunJw@mail.gmail.com>
	<698D3F1D-27D6-4FDB-9813-6414C11F650A@saxonica.com>
Message-ID: <CAP8fv=1rYXxm6ThaH5tW0t47-TNayYuE8FT3yHhchqKH-++Wew@mail.gmail.com>

Hello Michael,

2017-01-12 14:09 GMT+01:00 Michael Kay <mike at saxonica.com>:

> Logged here as a bug:
>
> https://www.w3.org/Bugs/Public/show_bug.cgi?id=30045


Thanks, will do it next time.

No, I think the subsequent points are far less important. The code used to
> specify the effect of the function is intended to be a specification, not
> an efficient implementation.
>

Agreed, but that wasn't my point. You may have the opinion that it wasn't
important, but I'm curious to know where anything tangible on laziness is
mentioned. As you say, not having any won't be very efficient, so you may
as well be explicit about it, right? I don't really understand why it's
preferable to have a syntax without an implementation, and I simply pointed
out that in the case of the fold-right example that becomes slightly odd...
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://x-query.com/pipermail/talk/attachments/20170112/3b7d3531/attachment.html>

From mike at saxonica.com  Thu Jan 12 08:27:37 2017
From: mike at saxonica.com (Michael Kay)
Date: Thu, 12 Jan 2017 16:27:37 +0000
Subject: [xquery-talk] Fold-right
In-Reply-To: <CAP8fv=1rYXxm6ThaH5tW0t47-TNayYuE8FT3yHhchqKH-++Wew@mail.gmail.com>
References: <CAP8fv=0gkmcc0-iYfyqvfW632kdOa8er4M+i+YKX5gZjquunJw@mail.gmail.com>
	<698D3F1D-27D6-4FDB-9813-6414C11F650A@saxonica.com>
	<CAP8fv=1rYXxm6ThaH5tW0t47-TNayYuE8FT3yHhchqKH-++Wew@mail.gmail.com>
Message-ID: <BACED20D-F5BA-48F5-8515-F3BEBA68B990@saxonica.com>

> 
> Agreed, but that wasn't my point. You may have the opinion that it wasn't important, but I'm curious to know where anything tangible on laziness is mentioned.

It isn't - deliberately. We leave "quality of implementation" issues entirely to the implementor. There are many implementation techniques available, including ones that may not have been invented yet, and there are different trade-offs between time and memory, and the spec quite deliberately doesn't get involved in such matters. The spec tells you what result to expect, it doesn't tell you when to expect it.

Michael Kay
Saxonica


> As you say, not having any won't be very efficient, so you may as well be explicit about it, right? I don't really understand why it's preferable to have a syntax without an implementation, and I simply pointed out that in the case of the fold-right example that becomes slightly odd...



From wshager at gmail.com  Thu Jan 12 09:10:14 2017
From: wshager at gmail.com (W.S. Hager)
Date: Thu, 12 Jan 2017 18:10:14 +0100
Subject: [xquery-talk] Fold-right
In-Reply-To: <BACED20D-F5BA-48F5-8515-F3BEBA68B990@saxonica.com>
References: <CAP8fv=0gkmcc0-iYfyqvfW632kdOa8er4M+i+YKX5gZjquunJw@mail.gmail.com>
	<698D3F1D-27D6-4FDB-9813-6414C11F650A@saxonica.com>
	<CAP8fv=1rYXxm6ThaH5tW0t47-TNayYuE8FT3yHhchqKH-++Wew@mail.gmail.com>
	<BACED20D-F5BA-48F5-8515-F3BEBA68B990@saxonica.com>
Message-ID: <CAP8fv=0jzct=Sh7jhQz5WB69d8w53MC=oVbpOjyjaNwCPa6Q=A@mail.gmail.com>

Can we please agree to add a footnote to the fold-right example that says
something along the lines of "this example algorithm may be optimized using
lazy evaluation in the interpreter"? I don't see how that is biasing the
spec toward a specific implementation...

2017-01-12 17:27 GMT+01:00 Michael Kay <mike at saxonica.com>:

> >
> > Agreed, but that wasn't my point. You may have the opinion that it
> wasn't important, but I'm curious to know where anything tangible on
> laziness is mentioned.
>
> It isn't - deliberately. We leave "quality of implementation" issues
> entirely to the implementor. There are many implementation techniques
> available, including ones that may not have been invented yet, and there
> are different trade-offs between time and memory, and the spec quite
> deliberately doesn't get involved in such matters. The spec tells you what
> result to expect, it doesn't tell you when to expect it.
>
> Michael Kay
> Saxonica
>
>
> > As you say, not having any won't be very efficient, so you may as well
> be explicit about it, right? I don't really understand why it's preferable
> to have a syntax without an implementation, and I simply pointed out that
> in the case of the fold-right example that becomes slightly odd...
>
>


-- 

W.S. Hager
Lagua Web Solutions
http://lagua.nl
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://x-query.com/pipermail/talk/attachments/20170112/448edd10/attachment.html>

From wshager at gmail.com  Thu Jan 12 09:21:56 2017
From: wshager at gmail.com (W.S. Hager)
Date: Thu, 12 Jan 2017 18:21:56 +0100
Subject: [xquery-talk] Fold-right
In-Reply-To: <CAP8fv=0jzct=Sh7jhQz5WB69d8w53MC=oVbpOjyjaNwCPa6Q=A@mail.gmail.com>
References: <CAP8fv=0gkmcc0-iYfyqvfW632kdOa8er4M+i+YKX5gZjquunJw@mail.gmail.com>
	<698D3F1D-27D6-4FDB-9813-6414C11F650A@saxonica.com>
	<CAP8fv=1rYXxm6ThaH5tW0t47-TNayYuE8FT3yHhchqKH-++Wew@mail.gmail.com>
	<BACED20D-F5BA-48F5-8515-F3BEBA68B990@saxonica.com>
	<CAP8fv=0jzct=Sh7jhQz5WB69d8w53MC=oVbpOjyjaNwCPa6Q=A@mail.gmail.com>
Message-ID: <CAP8fv=3LOD0YfcjvRa+Mrwt-U9uw5af0+DK6KHG3yh6V=qo_8Q@mail.gmail.com>

Better still, remove fold-right entirely, since laziness isn't part of the
spec (specifically infinite lists).

2017-01-12 18:10 GMT+01:00 W.S. Hager <wshager at gmail.com>:

> Can we please agree to add a footnote to the fold-right example that says
> something along the lines of "this example algorithm may be optimized using
> lazy evaluation in the interpreter"? I don't see how that is biasing the
> spec toward a specific implementation...
>
> 2017-01-12 17:27 GMT+01:00 Michael Kay <mike at saxonica.com>:
>
>> >
>> > Agreed, but that wasn't my point. You may have the opinion that it
>> wasn't important, but I'm curious to know where anything tangible on
>> laziness is mentioned.
>>
>> It isn't - deliberately. We leave "quality of implementation" issues
>> entirely to the implementor. There are many implementation techniques
>> available, including ones that may not have been invented yet, and there
>> are different trade-offs between time and memory, and the spec quite
>> deliberately doesn't get involved in such matters. The spec tells you what
>> result to expect, it doesn't tell you when to expect it.
>>
>> Michael Kay
>> Saxonica
>>
>>
>> > As you say, not having any won't be very efficient, so you may as well
>> be explicit about it, right? I don't really understand why it's preferable
>> to have a syntax without an implementation, and I simply pointed out that
>> in the case of the fold-right example that becomes slightly odd...
>>
>>
>
>
> --
>
> W.S. Hager
> Lagua Web Solutions
> http://lagua.nl
>



-- 

W.S. Hager
Lagua Web Solutions
http://lagua.nl
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://x-query.com/pipermail/talk/attachments/20170112/7cd92678/attachment.html>

From christian.gruen at gmail.com  Tue Jan 24 09:55:33 2017
From: christian.gruen at gmail.com (=?UTF-8?Q?Christian_Gr=C3=BCn?=)
Date: Tue, 24 Jan 2017 18:55:33 +0100
Subject: [xquery-talk] [ANN] BaseX 8.6: The Winter Edition
Message-ID: <CAP94bnNb8M4PUm7xZKHJNZuv=k-ighq3zUSY+aGu787QZKOvfg@mail.gmail.com>

Dear all,

BaseX 8.6 is out! The new version of our XML database system and
XQuery 3.1 processor includes countless improvements and
optimizations. Many of them have been triggered by your valuable
feedback, many others are the result of BaseX used in productive and
commercial environments.

The most prominent new features are:

 LOCKING
 - jobs without database access will never be locked
 - read transactions are now favored (adjustable via FAIRLOCK)

 RESTXQ
 - file monitoring was improved (adjustable via PARSERESTXQ)
 - authentication was reintroduced (no passwords anymore in web.xml)
 - id session attributes will show up in log data

 DBA
 - always accessible, even if job queue is full
 - pagination of table results

 INDEXING
 - path index improved: distinct values storage for numeric types

 XQUERY
 - aligned with latest version of XQuery 3.1
 - updated functions: map:find, map:merge, fn:sort, array:sort, ...
 - enhancements in User, Process, Jobs, REST and Database Module

 CSV DATA
 - improved import/export compatibility with Excel data

Visit http://basex.org to find the latest release, and check out
http://docs.basex.org/ to get more information. As always, we are
looking forward to your feedback. Enjoy!

Christian
BaseX Team