[subexp-daq] Nupeline

[Håkan T Johansson]

Dear Oliver and Günter,

just a suggestion, since you are at least two users of nupeline and may 
benefit from each other:

Perhaps it would be an idea to have the code available at e.g. an 
gitlab.com group nupeline (or something such).

At least the 'nupeline' repository has a license (MIT) that makes it 
possible to have it public.

For 'nupeline.configs' and 'nupeline.examples' this is however not clear. 
Perhaps Bastii can clarify that situation?

One could also think to only use parts of those repositories if there are 
parts that are unclear license-wise.

The reason for wanting also the examples is that they then can form the 
basis of CI testing, which would prevent unintentional breaking of them in 
the generic (nupeline) code.

Then you could also add the codes that you use locally as either more 
repositories, or as new parts of e.g. 'nupeline.examples' to again avoid 
generic changes to break them, by including them in CI builds.

I do not want to offer that I do anything here, but I could help with 
getting the CI working.  And possibly vet merge requests into a 'master' 
branch.  But that would not be maintenance, just based on look-and-feel 
and CI success.  It would be much better if someone that uses the code 
would do that job.  For starters, it sounds like were are not talking 
about much commits anyhow.

Who knows - nupeline might get a life of its own?  ;)

Cheers,
Håkan



On Fri, 22 Mar 2024, Oliver Papst wrote:

> Dear Günter,
>
> I am another user of nupeline. We use it both in Darmstadt for the DHIPS setup
> and at HIγS, Duke University, for our experiments using the Clover array setup.
>
> I have created several boxes myself, but I don’t have the capacity to put any
> work into nupeline myself. I just use it as-it-is. For lack of a better
> alternative that is known to us, we will continue to use it for now. I would
> also prefer using something that is actively maintained, though.
>
> Cheers,
>
> Oliver
>
> On Fri, 2024-03-22 at 16:49 +0000, Weber, Guenter Dr. wrote:
>> Dear Håkan,
>> 
>> 
>> thank you very much for the explanations.
>> 
>> 
>> To be independent from Bastian's legacy code we have now our tailor made unpacker for some of the LMD data. If we found a way to feed it not only with files from hard disk but also the data stream provided by UCESB, we could also use it for online analysis. We could then adapt this unpacker to the specific structure of our experimental data.
>> 
>> 
>> However, as UCESB itself is (as I understand) an unpacker for LMD data, probably the better way would be to use UCESB for basic unpacking into LMD events/subevents and then as a second stage a dedicated mapper (which in an ideal world would result from the settings of the modules in main.cfg) to interpret the data words. This is (again: as I understand it) the approach taken by Bastian. This second stage could then curate/organize the data and give it to subsequent programs in a more user-friendly data format than plain LMD structures.
>> 
>> 
>> However, I see little use to invest time into the existing NUPELINE package if Bastian was basicly the only person maintaining it. Is there maybe something else 'on the market'? Or am I mistaken and there is somebody out there, who is also continuing to use NUPELINE?
>> 
>> 
>> (Sorry, if this e-mail is a bit repetitive. I just want to make sure that my understanding of the situation is correct.)
>> 
>> 
>> 
>> 
>> Best greetings
>> 
>> Günter
>> 
>> 
>> 
>> 
>> ________________________________
>> Von: subexp-daq <subexp-daq-bounces at lists.chalmers.se> im Auftrag von Håkan T Johansson <f96hajo at chalmers.se>
>> Gesendet: Freitag, 22. März 2024 13:06
>> An: Discuss use of Nurdlib, TRLO II, drasi and UCESB.
>> Betreff: Re: [subexp-daq] Question on UCESB / UPEXPS
>> 
>> 
>> Dear Günter,
>> 
>> feel free to respond to issues separately, it was quite a bunch of
>> questions :-)  (I like!)
>> 
>> On Fri, 22 Mar 2024, Weber, Guenter Dr. wrote:
>> 
>> > 
>> > Dear friends,
>> > 
>> > 
>> > we have now checked the code for the SIS3316 module and will push a version
>> > with some bug fixes soon.
>> 
>> That is excellent news!
>> 
>> > The next thing on our list is to understand how we can access and store the
>> > data that we produce. Bastian told us the following command to write LMD
>> > files to hard disk:
>> > 
>> > 
>> > /LynxOS/mbsusr/mbsdaq/mbsrun/rio4/2021_mcalstruck/drasi/bin/lwrocctrl
>> > localhost --file-open=auto=1Gi,${file}
>> > 
>> > How does this LWROCCTRL work together with the Event Builder and UCESB which
>> > are started by the following commands?
>> > 
>> > ../drasi/bin/lwrocmerge \
>> >     --label=MCAL_EB \
>> >     --merge-mode=event \
>> >     --server=trans,flush=1 \
>> >     --server=stream,flush=1 \
>> >     --buf=size=800Mi \
>> >     --max-ev-size=10Mi \
>> >     --eb-master=rio4-mcal-2 \
>> >     --file-writer \
>> >     --drasi=rio4-mcal-2
>> > 
>> > ------------------------------------
>> 
>> lwrocctrl works by communicating with drasi instances.
>> for the lwrocctrl --file... options, it should be an instance which has a
>> --file-writer specified.
>> 
>> You can run the lwrocctrl also from another machine (e.g. your PC), but
>> then instead of 'localhost' give the name (or IP) of the machine where the
>> instance is running.
>> 
>> For the communication to work, both have to find a directory
>> ~/.drasi_tokens/ with one or more common files which it will do a cheap
>> hash of.  That is just some very minor protection against mistakes (users
>> talking with the wrong machine.  It is not a security measure as such.
>> 
>> To see if an instance is willing to talk, you can try
>> 
>> PATH-TO-DRASI/bin/bin/lwrocctrl --file-status [node]
>> 
>> > 
>> > while :
>> > do
>> > ~/mbsrun/rio4/mcalstruck/ucesb/empty/empty \
>> >     stream://localhost \
>> >     --server=stream:8001,bufsize=10Mi,flush=1,dataport:7001
>> > sleep 5
>> > done
>> 
>> That connects to the server above (lwrocmerge) started with
>> --server=stream.
>> 
>> And its --server option means it will act as a kind of proxy server.  I.e.
>> other users that want data can connect to that instead of directly to the
>> DAQ process.
>> 
>> > Moreover, to read in the online data stream as well as the LMD files from
>> > disk, to us it looks like Bastian used a programm called UCESB_IN which is
>> > using a mapping that is defined within UPEXPS to interpret the content of
>> > the LMD files. To us it looks like UCESB_IN is like a wrapper for UCESB to
>> > do a mapping of the content of the LMD events and to send out a data stream
>> > of some of this content. Attached please find the configuration file for
>> > UCESB_IN.
>> 
>> The attached .cfg file looks like a setup for Bastii's nupeline, which I
>> have very little clue about.
>> 
>> Indeed it sure looks like it in turn uses an UCESB unpacker.  With the
>> source in upexps.  That (upexps) has maintenance issues.
>> 
>> Perhaps someone else reading this mailing list is also using unpackers
>> from the upexps?  If so, now would be a good time to speak up, because I
>> think we need to figure out how to have something backwards compatible and
>> maintainable for you, without being 'encouraged' to do the maintenance
>> work that other users at GSI (R3B) refuse to do...
>> 
>> > Here, we are not sure up to which point there is a common approach to read
>> > experimental data and beyond that everybody is on his own.
>> 
>> ... while still sharing as much as possible with other users.
>> 
>> > Many thanks and best greetings from Jena
>> > Günter
>> 
>> Cheers,
>> Håkan
>
> -- 
> subexp-daq mailing list
> subexp-daq at lists.chalmers.se
> https://lists.chalmers.se/mailman/listinfo/subexp-daq