From mike at saxonica.com  Tue May  6 09:53:47 2014
From: mike at saxonica.com (Michael Kay)
Date: Tue, 6 May 2014 17:53:47 +0100
Subject: [xquery-talk] Random number generation : requirements
Message-ID: <845EBA9C-9B47-4263-92C2-05BDFBF7B635@saxonica.com>

The W3C WGs are looking at the idea of introducing a random-number function of some kind in XPath 3.1. The challenge of course is making this both usable and a pure function with no side-effects. We have various design ideas which we need to test against requirements.

If you have any applications that use or need such a function, please could I have a brief description of the way it uses random numbers, e.g. just wanting a single random number, a random permutation of 52 integers, an arbitrary sequence of random numebrs for test data generation, etc.

If you're currently using the EXSLT random-sequence() function, please share your experience with it.

Michael Kay
Saxonica

From matthias.brantner at 28msec.com  Tue May  6 10:42:23 2014
From: matthias.brantner at 28msec.com (Matthias Brantner)
Date: Tue, 6 May 2014 10:42:23 -0700
Subject: [xquery-talk] Random number generation : requirements
In-Reply-To: <845EBA9C-9B47-4263-92C2-05BDFBF7B635@saxonica.com>
References: <845EBA9C-9B47-4263-92C2-05BDFBF7B635@saxonica.com>
Message-ID: <B2341839-B0E1-4306-948E-6F0FF8E42981@28msec.com>

Michael

Here is the documentation of a module containing functions to generate
random numbers or strings with Zorba.

http://www.zorba.io/documentation/latest/modules/zorba/xdm/atomic/random

Please note that some functions are using Zorba's nondeterministic annotation.
Also note, that there is one function that creates uuids returned as string.

The functions have proven themselves useful in plenty of scenarios like
password or web session token generation.

I hope you find this module useful.

Matthias

On May 6, 2014, at 9:53 AM, Michael Kay <mike at saxonica.com> wrote:

> The W3C WGs are looking at the idea of introducing a random-number function of some kind in XPath 3.1. The challenge of course is making this both usable and a pure function with no side-effects. We have various design ideas which we need to test against requirements.
> 
> If you have any applications that use or need such a function, please could I have a brief description of the way it uses random numbers, e.g. just wanting a single random number, a random permutation of 52 integers, an arbitrary sequence of random numebrs for test data generation, etc.
> 
> If you're currently using the EXSLT random-sequence() function, please share your experience with it.
> 
> Michael Kay
> Saxonica
> _______________________________________________
> talk at x-query.com
> http://x-query.com/mailman/listinfo/talk



From christian.gruen at gmail.com  Tue May  6 10:51:21 2014
From: christian.gruen at gmail.com (=?ISO-8859-1?Q?Christian_Gr=FCn?=)
Date: Tue, 6 May 2014 19:51:21 +0200
Subject: [xquery-talk] Random number generation : requirements
In-Reply-To: <B2341839-B0E1-4306-948E-6F0FF8E42981@28msec.com>
References: <845EBA9C-9B47-4263-92C2-05BDFBF7B635@saxonica.com>
	<B2341839-B0E1-4306-948E-6F0FF8E42981@28msec.com>
Message-ID: <CAP94bnPYXADtFq1Au=JSS8isrNgp2cPv7barOwZkiX_=Z-gXBA@mail.gmail.com>

And here is the Random Module of BaseX:

  http://docs.basex.org/wiki/Random_Module

Similar to Zorba, most functions are non-deterministic. Is there any
particular reason why the official random functions need to be
deterministic?

Hope this helps,
Christian


On Tue, May 6, 2014 at 7:42 PM, Matthias Brantner
<matthias.brantner at 28msec.com> wrote:
> Michael
>
> Here is the documentation of a module containing functions to generate
> random numbers or strings with Zorba.
>
> http://www.zorba.io/documentation/latest/modules/zorba/xdm/atomic/random
>
> Please note that some functions are using Zorba's nondeterministic annotation.
> Also note, that there is one function that creates uuids returned as string.
>
> The functions have proven themselves useful in plenty of scenarios like
> password or web session token generation.
>
> I hope you find this module useful.
>
> Matthias
>
> On May 6, 2014, at 9:53 AM, Michael Kay <mike at saxonica.com> wrote:
>
>> The W3C WGs are looking at the idea of introducing a random-number function of some kind in XPath 3.1. The challenge of course is making this both usable and a pure function with no side-effects. We have various design ideas which we need to test against requirements.
>>
>> If you have any applications that use or need such a function, please could I have a brief description of the way it uses random numbers, e.g. just wanting a single random number, a random permutation of 52 integers, an arbitrary sequence of random numebrs for test data generation, etc.
>>
>> If you're currently using the EXSLT random-sequence() function, please share your experience with it.
>>
>> Michael Kay
>> Saxonica
>> _______________________________________________
>> talk at x-query.com
>> http://x-query.com/mailman/listinfo/talk
>
>
> _______________________________________________
> talk at x-query.com
> http://x-query.com/mailman/listinfo/talk


From mike at saxonica.com  Tue May  6 13:42:29 2014
From: mike at saxonica.com (Michael Kay)
Date: Tue, 6 May 2014 21:42:29 +0100
Subject: [xquery-talk] Random number generation : requirements
In-Reply-To: <CAP94bnPYXADtFq1Au=JSS8isrNgp2cPv7barOwZkiX_=Z-gXBA@mail.gmail.com>
References: <845EBA9C-9B47-4263-92C2-05BDFBF7B635@saxonica.com>
	<B2341839-B0E1-4306-948E-6F0FF8E42981@28msec.com>
	<CAP94bnPYXADtFq1Au=JSS8isrNgp2cPv7barOwZkiX_=Z-gXBA@mail.gmail.com>
Message-ID: <A3FDB62C-C08D-4A8B-9FD6-72203B8C172E@saxonica.com>

> 
> Similar to Zorba, most functions are non-deterministic. Is there any
> particular reason why the official random functions need to be
> deterministic?
> 

Yes, we don't have any machinery in the language semantics for declaring something as nondeterministic (easy to solve) or defining the semantics of how it should behave if thus annotated (much harder).

For example, if f() and g() are non-deterministic functions, how do we say in the spec that in the expression (f(), g()), f should be executed before g?

Michael Kay
Saxonica 



From christian.gruen at gmail.com  Tue May  6 13:52:42 2014
From: christian.gruen at gmail.com (=?ISO-8859-1?Q?Christian_Gr=FCn?=)
Date: Tue, 6 May 2014 22:52:42 +0200
Subject: [xquery-talk] Random number generation : requirements
In-Reply-To: <A3FDB62C-C08D-4A8B-9FD6-72203B8C172E@saxonica.com>
References: <845EBA9C-9B47-4263-92C2-05BDFBF7B635@saxonica.com>
	<B2341839-B0E1-4306-948E-6F0FF8E42981@28msec.com>
	<CAP94bnPYXADtFq1Au=JSS8isrNgp2cPv7barOwZkiX_=Z-gXBA@mail.gmail.com>
	<A3FDB62C-C08D-4A8B-9FD6-72203B8C172E@saxonica.com>
Message-ID: <CAP94bnNfrigcrwUWJK5s89Q4XXWgS=kTTFRQ0ECohxNiuPX_uQ@mail.gmail.com>

> Yes, we don't have any machinery in the language semantics for declaring something as nondeterministic (easy to solve) or defining the semantics of how it should behave if thus annotated (much harder).
>
> For example, if f() and g() are non-deterministic functions, how do we say in the spec that in the expression (f(), g()), f should be executed before g?

I see; so it seems to be related to questions (like the execution
order) that have also been discussed on the File Module

Thanks for the info,
Christian

From mike at saxonica.com  Tue May  6 15:40:26 2014
From: mike at saxonica.com (Michael Kay)
Date: Tue, 6 May 2014 23:40:26 +0100
Subject: [xquery-talk] Random number generation : requirements
In-Reply-To: <536962D7.20603@benibela.de>
References: <845EBA9C-9B47-4263-92C2-05BDFBF7B635@saxonica.com>
	<536962D7.20603@benibela.de>
Message-ID: <8FD4F5E3-AB92-4C25-9EAD-DB40EE453967@saxonica.com>

> 
> My policy on side effects is: all expressions containing side effects are going to be evaluated in order
> 

I do something like that in Saxon as well. But I don't attempt to define what "in order" means; for example, the order in which different global variables are evaluated. Doing this in the spec would be much more problematic.

Michael Kay
Saxonica



From msokolov at safaribooksonline.com  Tue May  6 15:48:18 2014
From: msokolov at safaribooksonline.com (Michael Sokolov)
Date: Tue, 06 May 2014 18:48:18 -0400
Subject: [xquery-talk] [xsl] Re: Random number generation : requirements
In-Reply-To: <20140506184036.44250@lists.mulberrytech.com>
References: <845EBA9C-9B47-4263-92C2-05BDFBF7B635@saxonica.com>
	<536962D7.20603@benibela.de>
	<20140506184036.44250@lists.mulberrytech.com>
Message-ID: <536966B2.2090302@safaribooksonline.com>

On 5/6/2014 6:41 PM, Michael Kay mike at saxonica.com wrote:
>> My policy on side effects is: all expressions containing side effects are going to be evaluated in order
>>
> I do something like that in Saxon as well. But I don't attempt to define what "in order" means; for example, the order in which different global variables are evaluated. Doing this in the spec would be much more problematic.
>
You don't think it would be reasonable to say something to the effect 
that the order in which non-deterministic expressions are evaluated is 
non-deterministic (ie implementation-defined)? Certainly it would be 
reasonable enough in the case of a random number generator.  Although I 
suppose if you are going to seed it, you would like the seed to effect 
the random numbers that are generated.

-Mike

From mike at saxonica.com  Tue May  6 15:58:43 2014
From: mike at saxonica.com (Michael Kay)
Date: Tue, 6 May 2014 23:58:43 +0100
Subject: [xquery-talk] [xsl] Re: Random number generation : requirements
In-Reply-To: <536966B2.2090302@safaribooksonline.com>
References: <845EBA9C-9B47-4263-92C2-05BDFBF7B635@saxonica.com>
	<536962D7.20603@benibela.de>
	<20140506184036.44250@lists.mulberrytech.com>
	<536966B2.2090302@safaribooksonline.com>
Message-ID: <4D32309C-BD28-44A5-A070-618E65D6C643@saxonica.com>


The big problem with a nondeterministic random() function is not defining the order of execution, but preventing it being optimised out of a loop. For example, how do we ensure that

$xxx[random() gt 0.5]

doesn't select either all the values or none?

Anyway, we're not planning to do non-determinism. This exercise is about designing a deterministic way to meet the requirement.

Michael Kay
Saxonica

On 6 May 2014, at 23:48, Michael Sokolov <msokolov at safaribooksonline.com> wrote:

> On 5/6/2014 6:41 PM, Michael Kay mike at saxonica.com wrote:
>>> My policy on side effects is: all expressions containing side effects are going to be evaluated in order
>>> 
>> I do something like that in Saxon as well. But I don't attempt to define what "in order" means; for example, the order in which different global variables are evaluated. Doing this in the spec would be much more problematic.
>> 
> You don't think it would be reasonable to say something to the effect that the order in which non-deterministic expressions are evaluated is non-deterministic (ie implementation-defined)? Certainly it would be reasonable enough in the case of a random number generator.  Although I suppose if you are going to seed it, you would like the seed to effect the random numbers that are generated.
> 
> -Mike
> _______________________________________________
> talk at x-query.com
> http://x-query.com/mailman/listinfo/talk



From msokolov at safaribooksonline.com  Tue May  6 16:15:56 2014
From: msokolov at safaribooksonline.com (Michael Sokolov)
Date: Tue, 06 May 2014 19:15:56 -0400
Subject: [xquery-talk] [xsl] Re: Random number generation : requirements
In-Reply-To: <4D32309C-BD28-44A5-A070-618E65D6C643@saxonica.com>
References: <845EBA9C-9B47-4263-92C2-05BDFBF7B635@saxonica.com>
	<536962D7.20603@benibela.de>
	<20140506184036.44250@lists.mulberrytech.com>
	<536966B2.2090302@safaribooksonline.com>
	<4D32309C-BD28-44A5-A070-618E65D6C643@saxonica.com>
Message-ID: <53696D2C.6040900@safaribooksonline.com>

My 2c:

I used an XQuery function based on Dmitry's version before; it works 
fine although it's a little inconvenient to have to keep passing in the 
prior value.

I would say the most convenient (or at least the most familiar) 
signature for a random function is random($n) returning a random number 
between 0 inclusive and $n exclusive; ideally it would return integers 
if $n is an integer, floating point numbers if $n is a floating point 
number, empty if $n is empty ? and an error otherwise.  And I would like 
a seed function.  Ideally this should be callable many times: I'm not 
sure how that could be done non-deterministically though.

I suppose a sequence would be useful, but it isn't the first thing that 
leaps to mind.  What if I'm not sure how many I'll need?

For example, one use case for me was to load a huge amount of data, and 
only include 1% of it, in order to generate a predictable test data 
sub-set. I want to write an XSLT template that returns nothing 99% of 
the time, and for the other 1% of the time it processed the content 
normally.  I want this to be based on an identifier in the content so 
that for a given seed, the same "random" 1% are selected each time: it 
should *not* be order-dependent, rather I would like to seed the random 
number generator with a hash of a given seed that is a configuration 
parameter, and a node-identifier, and then evaluate the next random 
number to see if it is > 0.01 (say).  Maybe there are other ways to do 
that, but that is what I did using Java.

-Mike


On 5/6/2014 6:58 PM, Michael Kay wrote:
> The big problem with a nondeterministic random() function is not defining the order of execution, but preventing it being optimised out of a loop. For example, how do we ensure that
>
> $xxx[random() gt 0.5]
>
> doesn't select either all the values or none?
>
> Anyway, we're not planning to do non-determinism. This exercise is about designing a deterministic way to meet the requirement.
>
> Michael Kay
> Saxonica
>
> On 6 May 2014, at 23:48, Michael Sokolov <msokolov at safaribooksonline.com> wrote:
>
>> On 5/6/2014 6:41 PM, Michael Kay mike at saxonica.com wrote:
>>>> My policy on side effects is: all expressions containing side effects are going to be evaluated in order
>>>>
>>> I do something like that in Saxon as well. But I don't attempt to define what "in order" means; for example, the order in which different global variables are evaluated. Doing this in the spec would be much more problematic.
>>>
>> You don't think it would be reasonable to say something to the effect that the order in which non-deterministic expressions are evaluated is non-deterministic (ie implementation-defined)? Certainly it would be reasonable enough in the case of a random number generator.  Although I suppose if you are going to seed it, you would like the seed to effect the random numbers that are generated.
>>
>> -Mike
>> _______________________________________________
>> talk at x-query.com
>> http://x-query.com/mailman/listinfo/talk


From leo at woerteler.de  Tue May  6 16:20:13 2014
From: leo at woerteler.de (=?UTF-8?B?TGVvbmFyZCBXw7ZydGVsZXI=?=)
Date: Wed, 07 May 2014 01:20:13 +0200
Subject: [xquery-talk] Random number generation : requirements
In-Reply-To: <845EBA9C-9B47-4263-92C2-05BDFBF7B635@saxonica.com>
References: <845EBA9C-9B47-4263-92C2-05BDFBF7B635@saxonica.com>
Message-ID: <53696E2D.1020705@woerteler.de>

Hello Michael,

Am 06.05.2014 18:53, schrieb Michael Kay:
> If you have any applications that use or need such a function, please could I have a brief description of the way it uses random numbers, e.g. just wanting a single random number, a random permutation of 52 integers, an arbitrary sequence of random numebrs for test data generation, etc.

in contrast to everyone else (it seems) I implemented a *deterministic* 
random-number generator in XQuery [1] in order to generate reproducible 
fuzzy tests for my higher-order xq-modules project [2].

I used a simple Linear Congruential Generator [3] and implemented the 
following functions:

     rng:with-random-ints(
       $seed as xs:integer,
       $n as xs:integer,
       $start as item()*,
       $f as function(item()*, item()*) as item()*
     )

realizes a constant-space fold over `$n` randomly generated `xs:int`s 
with initial seed `$seed`. This is used by

     rng:random-ints(
       $seed as xs:integer,
       $n as xs:integer
     ) as xs:int* {
       rng:with-random-ints($seed, $n, (), function($seq, $i) {
         ($seq, $i)
       })
     };

to generate a sequence of random `xs:int` values. The module is used in 
some tests [4,5].

You could obviously also implement `rng:with-random-ints(...)` in terms 
of `rng:random-ints(...)`:

     rng:with-random-ints($seed, $n, $start, $f) {
       fn:fold-left(rng:random-ints($seed, $n), $start, $f)
     };

-- Leo

[1] 
https://github.com/LeoWoerteler/xq-modules/blob/master/src/main/xquery/modules/rng.xqm
[2] https://github.com/LeoWoerteler/xq-modules
[3] http://en.wikipedia.org/wiki/Linear_congruential_generator
[4] 
https://github.com/LeoWoerteler/xq-modules/blob/master/src/test/xquery/int-set-test.xq#L33-51
[5] 
https://github.com/LeoWoerteler/xq-modules/blob/master/src/test/xquery/heap-test.xq#L10

From ghislain.fourny at 28msec.com  Wed May  7 04:13:29 2014
From: ghislain.fourny at 28msec.com (Ghislain Fourny)
Date: Wed, 7 May 2014 13:13:29 +0200
Subject: [xquery-talk] Random number generation : requirements
In-Reply-To: <53696E2D.1020705@woerteler.de>
References: <845EBA9C-9B47-4263-92C2-05BDFBF7B635@saxonica.com>
	<53696E2D.1020705@woerteler.de>
Message-ID: <CAPa8FbS8KUvSMvbcKQ6YkgHHJsQJ1sUNnXQhinQFU1JAZdPaqQ@mail.gmail.com>

Hi,

I think that Leo' s proposal, on a first read, is the most elegant way
I have seen so far that (pseudo-)randomness can be brought to XQuery
in a deterministic way: by exposing that it is actually
pseudo-randomness (seed) to the user.

In Zorba (not 3.0, but the latest, trunk revision, used by us), we
designed and implemented nondeterminism in a way fully integrated with
side effects. If this is of interest to the list(s), I can say a few
words about what we did. Of course, feedback is always appreciated.

The abstract model is as follows. Basically, a program is executed
against a sequence of snapshots. Each side effect jumps to a new
snapshot. Between snapshots, evaluation order is very important
("sequential"). Between side effects and within a snapshot though, it
is as if the engine were so fast that time did not elapse at all. Of
course no engine is that fast -- even though I could imagine Mike
proving me wrong here :-) -- but, joke aside. it can hide the elapse
of time from the user with, say, caching. This idea is crucial in our
design and was largely inspired by Michael Sperberg-McQueen's vision
in a discussion on functional languages a while ago.

For everything deterministic, this means that identical expressions
within the same snapshot and against the same dynamic context (mostly
variable bindings) always evaluate to the same result (except in the
case of newly created nodes where identity is new every time). Once a
side effect occurs and a new snapshot is reached, the result can again
be different (the cache gets reset).

By default, Zorba automatically caches every function call involving
atomics only, within a snapshot. If you ask many times for the current
time without side effects, it should give you the same answer. You can
also ask Zorba to "simulate" or "force" full determinism with XML or
JSON as well.

If a function, or transitively an expression, is stamped as
nondeterministic, it means in Zorba that it has to be evaluated every
time the specification semantics says it has to be. Within a snapshot
though, there is no evaluation order constraint and the implementation
is free to pick any order it wants.

Mike said:
> For example, if f() and g() are non-deterministic functions, how do we say in the spec that in the expression (f(), g()), f should be executed before g?
In Zorba, we don' t. It' s fine to evaluate g before f.
> $xxx[random() gt 0.5]
With our semantics, this should produce a new number every time,
because the spec says the predicate is evaluated for each item in the
lhs sequence. Evaluating the predicate only once is an optimization
that cannot be done here because of non-determinism.

On an abstract level, nondeterministim, to us (Zorba), is synonym with
the presence of (true/quantum) randomness, nothing less but also
nothing more.

Does it make sense? I hope I got the summary right.

Kind regards,
Ghislain


On Wed, May 7, 2014 at 1:20 AM, Leonard W?rteler <leo at woerteler.de> wrote:
> Hello Michael,
>
> Am 06.05.2014 18:53, schrieb Michael Kay:
>
>> If you have any applications that use or need such a function, please
>> could I have a brief description of the way it uses random numbers, e.g.
>> just wanting a single random number, a random permutation of 52 integers, an
>> arbitrary sequence of random numebrs for test data generation, etc.
>
>
> in contrast to everyone else (it seems) I implemented a *deterministic*
> random-number generator in XQuery [1] in order to generate reproducible
> fuzzy tests for my higher-order xq-modules project [2].
>
> I used a simple Linear Congruential Generator [3] and implemented the
> following functions:
>
>     rng:with-random-ints(
>       $seed as xs:integer,
>       $n as xs:integer,
>       $start as item()*,
>       $f as function(item()*, item()*) as item()*
>     )
>
> realizes a constant-space fold over `$n` randomly generated `xs:int`s with
> initial seed `$seed`. This is used by
>
>     rng:random-ints(
>       $seed as xs:integer,
>       $n as xs:integer
>     ) as xs:int* {
>       rng:with-random-ints($seed, $n, (), function($seq, $i) {
>         ($seq, $i)
>       })
>     };
>
> to generate a sequence of random `xs:int` values. The module is used in some
> tests [4,5].
>
> You could obviously also implement `rng:with-random-ints(...)` in terms of
> `rng:random-ints(...)`:
>
>     rng:with-random-ints($seed, $n, $start, $f) {
>       fn:fold-left(rng:random-ints($seed, $n), $start, $f)
>     };
>
> -- Leo
>
> [1]
> https://github.com/LeoWoerteler/xq-modules/blob/master/src/main/xquery/modules/rng.xqm
> [2] https://github.com/LeoWoerteler/xq-modules
> [3] http://en.wikipedia.org/wiki/Linear_congruential_generator
> [4]
> https://github.com/LeoWoerteler/xq-modules/blob/master/src/test/xquery/int-set-test.xq#L33-51
> [5]
> https://github.com/LeoWoerteler/xq-modules/blob/master/src/test/xquery/heap-test.xq#L10
> _______________________________________________
> talk at x-query.com
> http://x-query.com/mailman/listinfo/talk


From adam.retter at googlemail.com  Wed May  7 11:05:28 2014
From: adam.retter at googlemail.com (Adam Retter)
Date: Wed, 7 May 2014 19:05:28 +0100
Subject: [xquery-talk] Random number generation : requirements
In-Reply-To: <CAP94bnPYXADtFq1Au=JSS8isrNgp2cPv7barOwZkiX_=Z-gXBA@mail.gmail.com>
References: <845EBA9C-9B47-4263-92C2-05BDFBF7B635@saxonica.com>
	<B2341839-B0E1-4306-948E-6F0FF8E42981@28msec.com>
	<CAP94bnPYXADtFq1Au=JSS8isrNgp2cPv7barOwZkiX_=Z-gXBA@mail.gmail.com>
Message-ID: <CAPK0ichbUvz9b2y11SZ0YCX0wqL2s01MwM3WcKUsuqwXLQmPxw@mail.gmail.com>

Just to throw in the functions from eXist which are also non-deterministic -

http://www.exist-db.org/exist/apps/fundocs/index.html?action=search&type=name&q=random

Personally I would be more interested in having a good random number
source for generating strong UUIDs, at the moment we just have an
extension function which is a non-deterministic wrapper around some
Java - http://www.exist-db.org/exist/apps/fundocs/index.html?action=search&type=name&q=uuid

On 6 May 2014 18:51, Christian Gr?n <christian.gruen at gmail.com> wrote:
> And here is the Random Module of BaseX:
>
>   http://docs.basex.org/wiki/Random_Module
>
> Similar to Zorba, most functions are non-deterministic. Is there any
> particular reason why the official random functions need to be
> deterministic?
>
> Hope this helps,
> Christian
>
>
> On Tue, May 6, 2014 at 7:42 PM, Matthias Brantner
> <matthias.brantner at 28msec.com> wrote:
>> Michael
>>
>> Here is the documentation of a module containing functions to generate
>> random numbers or strings with Zorba.
>>
>> http://www.zorba.io/documentation/latest/modules/zorba/xdm/atomic/random
>>
>> Please note that some functions are using Zorba's nondeterministic annotation.
>> Also note, that there is one function that creates uuids returned as string.
>>
>> The functions have proven themselves useful in plenty of scenarios like
>> password or web session token generation.
>>
>> I hope you find this module useful.
>>
>> Matthias
>>
>> On May 6, 2014, at 9:53 AM, Michael Kay <mike at saxonica.com> wrote:
>>
>>> The W3C WGs are looking at the idea of introducing a random-number function of some kind in XPath 3.1. The challenge of course is making this both usable and a pure function with no side-effects. We have various design ideas which we need to test against requirements.
>>>
>>> If you have any applications that use or need such a function, please could I have a brief description of the way it uses random numbers, e.g. just wanting a single random number, a random permutation of 52 integers, an arbitrary sequence of random numebrs for test data generation, etc.
>>>
>>> If you're currently using the EXSLT random-sequence() function, please share your experience with it.
>>>
>>> Michael Kay
>>> Saxonica
>>> _______________________________________________
>>> talk at x-query.com
>>> http://x-query.com/mailman/listinfo/talk
>>
>>
>> _______________________________________________
>> talk at x-query.com
>> http://x-query.com/mailman/listinfo/talk
>
> _______________________________________________
> talk at x-query.com
> http://x-query.com/mailman/listinfo/talk



-- 
Adam Retter

skype: adam.retter
tweet: adamretter
http://www.adamretter.org.uk


From ghislain.fourny at 28msec.com  Thu May  8 06:34:14 2014
From: ghislain.fourny at 28msec.com (Ghislain Fourny)
Date: Thu, 8 May 2014 15:34:14 +0200
Subject: [xquery-talk] [xsl] Re: Random number generation : requirements
In-Reply-To: <536A97A6.8090808@benibela.de>
References: <845EBA9C-9B47-4263-92C2-05BDFBF7B635@saxonica.com>
	<536962D7.20603@benibela.de>
	<20140506184036.44250@lists.mulberrytech.com>
	<536966B2.2090302@safaribooksonline.com>
	<4D32309C-BD28-44A5-A070-618E65D6C643@saxonica.com>
	<536A97A6.8090808@benibela.de>
Message-ID: <CAPa8FbTPBvCG3sCaagrDOisuYu5z-Gxa2qZt+CbdvFMtdK_S8g@mail.gmail.com>

Hi Benito,

We had a discussion on monads not so long ago in the context of update
and scripting. I like the elegance of monads. It is my understanding.
though, that monads only defer side effects to until they are applied
-- and then you are back to the need for side effects semantics.

Somehow, side effects and their visibility to subsequent expressions
bring an irreducible amount of complexity to the design of the
language (evaluation order, snapshots, etc). I don' t think there is
an easy way to introduce them -- clean and elegant, maybe, but not
easy, or at least, I believe, not easier than what was done in the
current scripting draft or alternate implementations.

That said, I think it is an important point that non-determinism is
distinct from side effects. You can have deterministic side effects,
and you can have non-determinism without side effects (as I tried to
illustrate in my former e-mail). So far, I am not convinced that
introducing side effects would address non-determinism.

Does it make sense?

Kind regards,
Ghislain


On Wed, May 7, 2014 at 10:29 PM, Benito van der Zander
<benito at benibela.de> wrote:
> Hi Michael,
> perhaps it is time to add monads to XPath?
>
> They should solve this issue and all related ones
>
>
>
> Best,
> Benito
>
>
> On 05/07/2014 12:58 AM, Michael Kay wrote:
>
> The big problem with a nondeterministic random() function is not defining
> the order of execution, but preventing it being optimised out of a loop. For
> example, how do we ensure that
>
> $xxx[random() gt 0.5]
>
> doesn't select either all the values or none?
>
> Anyway, we're not planning to do non-determinism. This exercise is about
> designing a deterministic way to meet the requirement.
>
> Michael Kay
> Saxonica
>
> On 6 May 2014, at 23:48, Michael Sokolov <msokolov at safaribooksonline.com>
> wrote:
>
> On 5/6/2014 6:41 PM, Michael Kay mike at saxonica.com wrote:
>
> My policy on side effects is: all expressions containing side effects are
> going to be evaluated in order
>
> I do something like that in Saxon as well. But I don't attempt to define
> what "in order" means; for example, the order in which different global
> variables are evaluated. Doing this in the spec would be much more
> problematic.
>
> You don't think it would be reasonable to say something to the effect that
> the order in which non-deterministic expressions are evaluated is
> non-deterministic (ie implementation-defined)? Certainly it would be
> reasonable enough in the case of a random number generator.  Although I
> suppose if you are going to seed it, you would like the seed to effect the
> random numbers that are generated.
>
> -Mike
> _______________________________________________
> talk at x-query.com
> http://x-query.com/mailman/listinfo/talk
>
>
>
>
> _______________________________________________
> talk at x-query.com
> http://x-query.com/mailman/listinfo/talk

From ihe.onwuka at gmail.com  Sat May 24 00:05:32 2014
From: ihe.onwuka at gmail.com (Ihe Onwuka)
Date: Sat, 24 May 2014 08:05:32 +0100
Subject: [xquery-talk] Grouping Scope and counters
Message-ID: <CALfs7+w4cT9O-OahbDhYNV4f4g5pqXT=VNPfLGb=uKg2hi1FKQ@mail.gmail.com>

for $x at $i in blah
  group by $x/@a
   return
     if ($i mod 10000 eq 0) then ..... else .....


Now the problem is that XQuery (of the eXist variety) denies knowledge of
the existence of the $i variable in the return clause. Presumably this is
because the group construct interferes with the lexical scoping.

I do not have to do this with XQuery as I can solve it downstream but I
would be interested in learning more about this scenario.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://x-query.com/pipermail/talk/attachments/20140524/798dbcf6/attachment.html>

From wolfgang at exist-db.org  Sat May 24 03:32:16 2014
From: wolfgang at exist-db.org (wolfgang at exist-db.org)
Date: Sat, 24 May 2014 12:32:16 +0200
Subject: [xquery-talk] Grouping Scope and counters
In-Reply-To: <CALfs7+w4cT9O-OahbDhYNV4f4g5pqXT=VNPfLGb=uKg2hi1FKQ@mail.gmail.com>
References: <CALfs7+w4cT9O-OahbDhYNV4f4g5pqXT=VNPfLGb=uKg2hi1FKQ@mail.gmail.com>
Message-ID: <EE1435D7-C977-4AE3-B176-942AAB2A3D82@exist-db.org>

> Now the problem is that XQuery (of the eXist variety) denies knowledge of the existence of the $i variable in the return clause.

Yes, the original implementor left a todo comment in the code ;-) The $i variable should be in scope. I?ve just submitted a pull request for this.

Wolfgang

From Inderjeet.Narang at NIIT-Tech.com  Thu May 29 03:07:09 2014
From: Inderjeet.Narang at NIIT-Tech.com (Inderjeet Singh Narang)
Date: Thu, 29 May 2014 10:07:09 +0000
Subject: [xquery-talk] generate and schedule report
Message-ID: <B1F885F78CB55942A43A4DEF50F16FCFC7FBF801@NOIDA-TZ-MSG01.IN.NIIT-Tech.com>

I am new to xquery. I want to generate excel report through scheduler. Please help me to sort out the issue.
Your help will be greately appreciated.

Thanks & Regards
Inderjeet







DISCLAIMER

The information transmitted, including any attachments, is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of the author of this e-mail is strictly prohibited. If you receive this email by mistake, please advise the sender and delete it immediately. Email transmission cannot be guaranteed to be secure, or error free as information could be intercepted, corrupted, lost or destroyed as a result of the transmission process. Views or opinions presented in this email are solely those of the author and do not necessarily represent those of the company. Please be aware that the Company monitors email communications through our networks for regulatory compliance purposes. NIIT Technologies Limited and its related entities consider the privacy of its clients to be of utmost importance and works to protect it. The Privacy Policy of the Company can be accessed from our website ?www.niit-tech.com. (s).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://x-query.com/pipermail/talk/attachments/20140529/7b870e29/attachment.html>

From Inderjeet.Narang at NIIT-Tech.com  Thu May 29 09:38:07 2014
From: Inderjeet.Narang at NIIT-Tech.com (Inderjeet Singh Narang)
Date: Thu, 29 May 2014 16:38:07 +0000
Subject: [xquery-talk] generate and schedule report
In-Reply-To: <19480ABD-EF15-4438-AAED-2036D5C53923@gmail.com>
References: <B1F885F78CB55942A43A4DEF50F16FCFC7FBF801@NOIDA-TZ-MSG01.IN.NIIT-Tech.com>
	<F92BA4B1-A0A9-499B-ACC9-315B84FF8E19@ptsem.edu>,
	<19480ABD-EF15-4438-AAED-2036D5C53923@gmail.com>
Message-ID: <B1F885F78CB55942A43A4DEF50F16FCFC7FBF85B@NOIDA-TZ-MSG01.IN.NIIT-Tech.com>

Hi,
I am using eXist-db and easydita.

I want to fetch records from eXist-db and then generate excel report through scheduler at a specified time say 1:00 PM everyday. At 1:00 PM scheduler will run and will generate excel report by fetching data from eXist-db.

Thanks
Inderjeet
________________________________
From: Loren Cahlander [loren.cahlander at gmail.com]
Sent: Thursday, May 29, 2014 8:53 PM
To: Murray, Gregory
Cc: Inderjeet Singh Narang; talk at x-query.com
Subject: Re: [xquery-talk] generate and schedule report

What system are you using?  I am thinking eXist-db due to the scheduler reference.

Sent from my iPad

On May 29, 2014, at 8:41 AM, "Murray, Gregory" <gregory.murray at ptsem.edu<mailto:gregory.murray at ptsem.edu>> wrote:

Please post a short example of what the input looks like and what of the format you want to output. Alternatively, it sounds like you are converting one form of XML to another, in which case XSLT might be a simpler option than XQuery.


On May 29, 2014, at 6:07 AM, Inderjeet Singh Narang wrote:

I am new to xquery. I want to generate excel report through scheduler. Please help me to sort out the issue.
Your help will be greately appreciated.

Thanks & Regards
Inderjeet

_______________________________________________
talk at x-query.com<mailto:talk at x-query.com>
http://x-query.com/mailman/listinfo/talk

_______________________________________________
talk at x-query.com<mailto:talk at x-query.com>
http://x-query.com/mailman/listinfo/talk






DISCLAIMER

The information transmitted, including any attachments, is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of the author of this e-mail is strictly prohibited. If you receive this email by mistake, please advise the sender and delete it immediately. Email transmission cannot be guaranteed to be secure, or error free as information could be intercepted, corrupted, lost or destroyed as a result of the transmission process. Views or opinions presented in this email are solely those of the author and do not necessarily represent those of the company. Please be aware that the Company monitors email communications through our networks for regulatory compliance purposes. NIIT Technologies Limited and its related entities consider the privacy of its clients to be of utmost importance and works to protect it. The Privacy Policy of the Company can be accessed from our website ?www.niit-tech.com. (s).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://x-query.com/pipermail/talk/attachments/20140529/1194143b/attachment.html>

From joewiz at gmail.com  Thu May 29 10:56:59 2014
From: joewiz at gmail.com (Joe Wicentowski)
Date: Thu, 29 May 2014 13:56:59 -0400
Subject: [xquery-talk] generate and schedule report
In-Reply-To: <796B539D-8982-49F6-92EA-DEFDB36247EC@gmail.com>
References: <B1F885F78CB55942A43A4DEF50F16FCFC7FBF801@NOIDA-TZ-MSG01.IN.NIIT-Tech.com>
	<F92BA4B1-A0A9-499B-ACC9-315B84FF8E19@ptsem.edu>
	<19480ABD-EF15-4438-AAED-2036D5C53923@gmail.com>
	<B1F885F78CB55942A43A4DEF50F16FCFC7FBF85B@NOIDA-TZ-MSG01.IN.NIIT-Tech.com>
	<796B539D-8982-49F6-92EA-DEFDB36247EC@gmail.com>
Message-ID: <CAHwerk1rgPO58koNLan==ZVFnpK4MxmpfhmupZfSwCW2_sMP7A@mail.gmail.com>

> I do not know of any module to generate an excel spreadsheet within eXist.  You can read a spreadsheet using the content extraction function module.

Pete Aven's article on generating Excel documents with XQuery is
straightforward to apply in eXist-db:

  http://developer.marklogic.com/blog/smallchanges/2007-12-04

(His whole series of articles is worth reading if you're interested in
generating various Office documents with XQuery.)

Joe

From loren.cahlander at gmail.com  Thu May 29 11:43:40 2014
From: loren.cahlander at gmail.com (Loren Cahlander)
Date: Thu, 29 May 2014 14:43:40 -0400
Subject: [xquery-talk] generate and schedule report
In-Reply-To: <CAHwerk1rgPO58koNLan==ZVFnpK4MxmpfhmupZfSwCW2_sMP7A@mail.gmail.com>
References: <B1F885F78CB55942A43A4DEF50F16FCFC7FBF801@NOIDA-TZ-MSG01.IN.NIIT-Tech.com>
	<F92BA4B1-A0A9-499B-ACC9-315B84FF8E19@ptsem.edu>
	<19480ABD-EF15-4438-AAED-2036D5C53923@gmail.com>
	<B1F885F78CB55942A43A4DEF50F16FCFC7FBF85B@NOIDA-TZ-MSG01.IN.NIIT-Tech.com>
	<796B539D-8982-49F6-92EA-DEFDB36247EC@gmail.com>
	<CAHwerk1rgPO58koNLan==ZVFnpK4MxmpfhmupZfSwCW2_sMP7A@mail.gmail.com>
Message-ID: <A0DD3002-7DBA-4125-98F8-ABBA78918688@gmail.com>

Thanks Joe!

Sent from my iPad

On May 29, 2014, at 1:56 PM, Joe Wicentowski <joewiz at gmail.com> wrote:

>> I do not know of any module to generate an excel spreadsheet within eXist.  You can read a spreadsheet using the content extraction function module.
> 
> Pete Aven's article on generating Excel documents with XQuery is
> straightforward to apply in eXist-db:
> 
>  http://developer.marklogic.com/blog/smallchanges/2007-12-04
> 
> (His whole series of articles is worth reading if you're interested in
> generating various Office documents with XQuery.)
> 
> Joe


From Inderjeet.Narang at NIIT-Tech.com  Fri May 30 03:36:06 2014
From: Inderjeet.Narang at NIIT-Tech.com (Inderjeet Singh Narang)
Date: Fri, 30 May 2014 10:36:06 +0000
Subject: [xquery-talk] generate and schedule report
In-Reply-To: <A0DD3002-7DBA-4125-98F8-ABBA78918688@gmail.com>
References: <B1F885F78CB55942A43A4DEF50F16FCFC7FBF801@NOIDA-TZ-MSG01.IN.NIIT-Tech.com>
	<F92BA4B1-A0A9-499B-ACC9-315B84FF8E19@ptsem.edu>
	<19480ABD-EF15-4438-AAED-2036D5C53923@gmail.com>
	<B1F885F78CB55942A43A4DEF50F16FCFC7FBF85B@NOIDA-TZ-MSG01.IN.NIIT-Tech.com>
	<796B539D-8982-49F6-92EA-DEFDB36247EC@gmail.com>
	<CAHwerk1rgPO58koNLan==ZVFnpK4MxmpfhmupZfSwCW2_sMP7A@mail.gmail.com>,
	<A0DD3002-7DBA-4125-98F8-ABBA78918688@gmail.com>
Message-ID: <B1F885F78CB55942A43A4DEF50F16FCFC7FBF8B0@NOIDA-TZ-MSG01.IN.NIIT-Tech.com>

Thanks for all your suggestions.
I am able to generate excel report as mentioned here http://jsfiddle.net/lesson8/wVejP/. This report get generated when button is clicked.
Now i want to generate the report through scheduler as scheduler is not going to click on button so I am stuck here.
any suggestion would be appreciated.

Thanks
Inderjeet
________________________________________
From: Loren Cahlander [loren.cahlander at gmail.com]
Sent: Friday, May 30, 2014 12:13 AM
To: Joe Wicentowski
Cc: Inderjeet Singh Narang; talk at x-query.com; Murray, Gregory
Subject: Re: [xquery-talk] generate and schedule report

Thanks Joe!

Sent from my iPad

On May 29, 2014, at 1:56 PM, Joe Wicentowski <joewiz at gmail.com> wrote:

>> I do not know of any module to generate an excel spreadsheet within eXist.  You can read a spreadsheet using the content extraction function module.
>
> Pete Aven's article on generating Excel documents with XQuery is
> straightforward to apply in eXist-db:
>
>  http://developer.marklogic.com/blog/smallchanges/2007-12-04
>
> (His whole series of articles is worth reading if you're interested in
> generating various Office documents with XQuery.)
>
> Joe






DISCLAIMER

The information transmitted, including any attachments, is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of the author of this e-mail is strictly prohibited. If you receive this email by mistake, please advise the sender and delete it immediately. Email transmission cannot be guaranteed to be secure, or error free as information could be intercepted, corrupted, lost or destroyed as a result of the transmission process. Views or opinions presented in this email are solely those of the author and do not necessarily represent those of the company. Please be aware that the Company monitors email communications through our networks for regulatory compliance purposes. NIIT Technologies Limited and its related entities consider the privacy of its clients to be of utmost importance and works to protect it. The Privacy Policy of the Company can be accessed from our website ?www.niit-tech.com. (s).