[subexp-daq] Question on UCESB / UPEXPS

[Håkan T Johansson]

On Fri, 22 Mar 2024, Weber, Guenter Dr. wrote:

> 
> Dear Håkan,
> 
> 
> thank you very much for the explanations.
> 
> 
> To be independent from Bastian's legacy code we have now our tailor made
> unpacker for some of the LMD data. If we found a way to feed it not only
> with files from hard disk but also the data stream provided by UCESB, we
> could also use it for online analysis. We could then adapt this unpacker to
> the specific structure of our experimental data.

One could use ucesb in some kind of pass-through mode to read data from a 
network source.  Roughly:

empty/empty --stream=HOST:PORT --output=- | ./other_unpacker_program

> However, as UCESB itself is (as I understand) an unpacker for LMD data,
> probably the better way would be to use UCESB for basic unpacking into LMD

UCESB is maily used to unpack _from_ LMD files.  (The LMD -> LMD 
gymnastics that it can do are more like additions that turned out to be 
useful.)

The UCESB empty/empty program is a compile of ucesb without any data 
mapping, so it only 'exposes' the LMD->LMD features.  In the above case, 
just reading data from LMD stream (or transport) servers, and delivering 
it on stdout.

> events/subevents and then as a second stage a dedicated mapper (which in an
> ideal world would result from the settings of the modules in main.cfg) to
> interpret the data words. This is (again: as I understand it) the approach
> taken by Bastian. This second stage could then curate/organize the data and
> give it to subsequent programs in a more user-friendly data format than
> plain LMD structures.

I think the more plain data structures that Bastii uses in e.g. nupeline 
(as referenced in ucesb_in.cfg) are from a UCESB-based unpacker, with the 
specifications (and mappings) in the upexps repository.

> However, I see little use to invest time into the existing NUPELINE package
> if Bastian was basicly the only person maintaining it. Is there maybe
> something else 'on the market'? Or am I mistaken and there is somebody out
> there, who is also continuing to use NUPELINE?
> 
> 
> (Sorry, if this e-mail is a bit repetitive. I just want to make sure that my
> understanding of the situation is correct.)
> 
> 
> 
> 
> Best greetings
> 
> Günter


Cheers,
Håkan



> 
> 
> 
> 
> 
> ____________________________________________________________________________
> Von: subexp-daq <subexp-daq-bounces at lists.chalmers.se> im Auftrag von Håkan
> T Johansson <f96hajo at chalmers.se>
> Gesendet: Freitag, 22. März 2024 13:06
> An: Discuss use of Nurdlib, TRLO II, drasi and UCESB.
> Betreff: Re: [subexp-daq] Question on UCESB / UPEXPS  
> 
> Dear Günter,
> 
> feel free to respond to issues separately, it was quite a bunch of
> questions :-)  (I like!)
> 
> On Fri, 22 Mar 2024, Weber, Guenter Dr. wrote:
> 
> >
> > Dear friends,
> >
> >
> > we have now checked the code for the SIS3316 module and will push a
> version
> > with some bug fixes soon.
> 
> That is excellent news!
> 
> > The next thing on our list is to understand how we can access and store
> the
> > data that we produce. Bastian told us the following command to write LMD
> > files to hard disk:
> >
> >
> > /LynxOS/mbsusr/mbsdaq/mbsrun/rio4/2021_mcalstruck/drasi/bin/lwrocctrl
> > localhost --file-open=auto=1Gi,${file}
> >
> > How does this LWROCCTRL work together with the Event Builder and UCESB
> which
> > are started by the following commands?
> >
> > ../drasi/bin/lwrocmerge \
> >     --label=MCAL_EB \
> >     --merge-mode=event \
> >     --server=trans,flush=1 \
> >     --server=stream,flush=1 \
> >     --buf=size=800Mi \
> >     --max-ev-size=10Mi \
> >     --eb-master=rio4-mcal-2 \
> >     --file-writer \
> >     --drasi=rio4-mcal-2
> >
> > ------------------------------------
> 
> lwrocctrl works by communicating with drasi instances.
> for the lwrocctrl --file... options, it should be an instance which has a
> --file-writer specified.
> 
> You can run the lwrocctrl also from another machine (e.g. your PC), but
> then instead of 'localhost' give the name (or IP) of the machine where the
> instance is running.
> 
> For the communication to work, both have to find a directory
> ~/.drasi_tokens/ with one or more common files which it will do a cheap
> hash of.  That is just some very minor protection against mistakes (users
> talking with the wrong machine.  It is not a security measure as such.
> 
> To see if an instance is willing to talk, you can try
> 
> PATH-TO-DRASI/bin/bin/lwrocctrl --file-status [node]
> 
> >
> > while :
> > do
> > ~/mbsrun/rio4/mcalstruck/ucesb/empty/empty \
> >     stream://localhost \
> >     --server=stream:8001,bufsize=10Mi,flush=1,dataport:7001
> > sleep 5
> > done    
> 
> That connects to the server above (lwrocmerge) started with
> --server=stream.
> 
> And its --server option means it will act as a kind of proxy server.  I.e.
> other users that want data can connect to that instead of directly to the
> DAQ process.
> 
> > Moreover, to read in the online data stream as well as the LMD files from
> > disk, to us it looks like Bastian used a programm called UCESB_IN which is
> > using a mapping that is defined within UPEXPS to interpret the content of
> > the LMD files. To us it looks like UCESB_IN is like a wrapper for UCESB to
> > do a mapping of the content of the LMD events and to send out a data
> stream
> > of some of this content. Attached please find the configuration file for
> > UCESB_IN.
> 
> The attached .cfg file looks like a setup for Bastii's nupeline, which I
> have very little clue about.
> 
> Indeed it sure looks like it in turn uses an UCESB unpacker.  With the
> source in upexps.  That (upexps) has maintenance issues.
> 
> Perhaps someone else reading this mailing list is also using unpackers
> from the upexps?  If so, now would be a good time to speak up, because I
> think we need to figure out how to have something backwards compatible and
> maintainable for you, without being 'encouraged' to do the maintenance
> work that other users at GSI (R3B) refuse to do...
> 
> > Here, we are not sure up to which point there is a common approach to read
> > experimental data and beyond that everybody is on his own.
> 
> ... while still sharing as much as possible with other users.
> 
> > Many thanks and best greetings from Jena
> > Günter
> 
> Cheers,
> Håkan
> 
>