[subexp-daq] Nupeline

[Håkan T Johansson]

Hi Bastii,

How about gitlab.com in a group nupeline ?

Then it can re-use the CI scripts that exist already and at least I have 
more experience with gitlab CI.

Besides, github actions (their CI) gave me the creeps when I tried 
something with r3broot once.  By default they were allowed to modify my 
repositories!

Cheers,
Håkan



On Thu, 4 Apr 2024, Loeher, Bastian Dr. wrote:

> Hi everyone,
> I could make all related repositories freely available and put respective
> licenses on the repositories where they are missing. None of the code has
> unclear origins.
> 
> Where would be a good place? I could even put it up on github.com.
> 
> Cheers
> Bastii
> 
> Am 04.04.2024 07:02 schrieb Håkan T Johansson <f96hajo at chalmers.se>:
> 
> Dear Oliver and Günter,
> 
> just a suggestion, since you are at least two users of nupeline and may
> benefit from each other:
> 
> Perhaps it would be an idea to have the code available at e.g. an
> gitlab.com group nupeline (or something such).
> 
> At least the 'nupeline' repository has a license (MIT) that makes it
> possible to have it public.
> 
> For 'nupeline.configs' and 'nupeline.examples' this is however not clear.
> Perhaps Bastii can clarify that situation?
> 
> One could also think to only use parts of those repositories if there are
> parts that are unclear license-wise.
> 
> The reason for wanting also the examples is that they then can form the
> basis of CI testing, which would prevent unintentional breaking of them in
> the generic (nupeline) code.
> 
> Then you could also add the codes that you use locally as either more
> repositories, or as new parts of e.g. 'nupeline.examples' to again avoid
> generic changes to break them, by including them in CI builds.
> 
> I do not want to offer that I do anything here, but I could help with
> getting the CI working.  And possibly vet merge requests into a 'master'
> branch.  But that would not be maintenance, just based on look-and-feel
> and CI success.  It would be much better if someone that uses the code
> would do that job.  For starters, it sounds like were are not talking
> about much commits anyhow.
> 
> Who knows - nupeline might get a life of its own?  ;)
> 
> Cheers,
> Håkan
> 
> 
> 
> On Fri, 22 Mar 2024, Oliver Papst wrote:
> 
> > Dear Günter,
> >
> > I am another user of nupeline. We use it both in Darmstadt for the DHIPS
> setup
> > and at HIγS, Duke University, for our experiments using the Clover array
> setup.
> >
> > I have created several boxes myself, but I don’t have the capacity to put
> any
> > work into nupeline myself. I just use it as-it-is. For lack of a better
> > alternative that is known to us, we will continue to use it for now. I
> would
> > also prefer using something that is actively maintained, though.
> >
> > Cheers,
> >
> > Oliver
> >
> > On Fri, 2024-03-22 at 16:49 +0000, Weber, Guenter Dr. wrote:
> >> Dear Håkan,
> >>
> >>
> >> thank you very much for the explanations.
> >>
> >>
> >> To be independent from Bastian's legacy code we have now our tailor made
> unpacker for some of the LMD data. If we found a way to feed it not only
> with files from hard disk but also the data stream provided by UCESB, we
> could also use it for online analysis. We could then adapt this unpacker to
> the specific structure of our experimental data.
> >>
> >>
> >> However, as UCESB itself is (as I understand) an unpacker for LMD data,
> probably the better way would be to use UCESB for basic unpacking into LMD
> events/subevents and then as a second stage a dedicated mapper (which in an
> ideal world would result from the settings of the modules in main.cfg) to
> interpret the data words. This is (again: as I understand it) the approach
> taken by Bastian. This second stage could then curate/organize the data and
> give it to subsequent programs in a more user-friendly data format than
> plain LMD structures.
> >>
> >>
> >> However, I see little use to invest time into the existing NUPELINE
> package if Bastian was basicly the only person maintaining it. Is there
> maybe something else 'on the market'? Or am I mistaken and there is somebody
> out there, who is also continuing to use NUPELINE?
> >>
> >>
> >> (Sorry, if this e-mail is a bit repetitive. I just want to make sure that
> my understanding of the situation is correct.)
> >>
> >>
> >>
> >>
> >> Best greetings
> >>
> >> Günter
> >>
> >>
> >>
> >>
> >> ________________________________
> >> Von: subexp-daq <subexp-daq-bounces at lists.chalmers.se> im Auftrag von
> Håkan T Johansson <f96hajo at chalmers.se>
> >> Gesendet: Freitag, 22. März 2024 13:06
> >> An: Discuss use of Nurdlib, TRLO II, drasi and UCESB.
> >> Betreff: Re: [subexp-daq] Question on UCESB / UPEXPS
> >>
> >>
> >> Dear Günter,
> >>
> >> feel free to respond to issues separately, it was quite a bunch of
> >> questions :-)  (I like!)
> >>
> >> On Fri, 22 Mar 2024, Weber, Guenter Dr. wrote:
> >>
> >> >
> >> > Dear friends,
> >> >
> >> >
> >> > we have now checked the code for the SIS3316 module and will push a
> version
> >> > with some bug fixes soon.
> >>
> >> That is excellent news!
> >>
> >> > The next thing on our list is to understand how we can access and store
> the
> >> > data that we produce. Bastian told us the following command to write
> LMD
> >> > files to hard disk:
> >> >
> >> >
> >> > /LynxOS/mbsusr/mbsdaq/mbsrun/rio4/2021_mcalstruck/drasi/bin/lwrocctrl
> >> > localhost --file-open=auto=1Gi,${file}
> >> >
> >> > How does this LWROCCTRL work together with the Event Builder and UCESB
> which
> >> > are started by the following commands?
> >> >
> >> > ../drasi/bin/lwrocmerge \
> >> >     --label=MCAL_EB \
> >> >     --merge-mode=event \
> >> >     --server=trans,flush=1 \
> >> >     --server=stream,flush=1 \
> >> >     --buf=size=800Mi \
> >> >     --max-ev-size=10Mi \
> >> >     --eb-master=rio4-mcal-2 \
> >> >     --file-writer \
> >> >     --drasi=rio4-mcal-2
> >> >
> >> > ------------------------------------
> >>
> >> lwrocctrl works by communicating with drasi instances.
> >> for the lwrocctrl --file... options, it should be an instance which has a
> >> --file-writer specified.
> >>
> >> You can run the lwrocctrl also from another machine (e.g. your PC), but
> >> then instead of 'localhost' give the name (or IP) of the machine where
> the
> >> instance is running.
> >>
> >> For the communication to work, both have to find a directory
> >> ~/.drasi_tokens/ with one or more common files which it will do a cheap
> >> hash of.  That is just some very minor protection against mistakes (users
> >> talking with the wrong machine.  It is not a security measure as such.
> >>
> >> To see if an instance is willing to talk, you can try
> >>
> >> PATH-TO-DRASI/bin/bin/lwrocctrl --file-status [node]
> >>
> >> >
> >> > while :
> >> > do
> >> > ~/mbsrun/rio4/mcalstruck/ucesb/empty/empty \
> >> >     stream://localhost \
> >> >     --server=stream:8001,bufsize=10Mi,flush=1,dataport:7001
> >> > sleep 5
> >> > done
> >>
> >> That connects to the server above (lwrocmerge) started with
> >> --server=stream.
> >>
> >> And its --server option means it will act as a kind of proxy server. 
> I.e.
> >> other users that want data can connect to that instead of directly to the
> >> DAQ process.
> >>
> >> > Moreover, to read in the online data stream as well as the LMD files
> from
> >> > disk, to us it looks like Bastian used a programm called UCESB_IN which
> is
> >> > using a mapping that is defined within UPEXPS to interpret the content
> of
> >> > the LMD files. To us it looks like UCESB_IN is like a wrapper for UCESB
> to
> >> > do a mapping of the content of the LMD events and to send out a data
> stream
> >> > of some of this content. Attached please find the configuration file
> for
> >> > UCESB_IN.
> >>
> >> The attached .cfg file looks like a setup for Bastii's nupeline, which I
> >> have very little clue about.
> >>
> >> Indeed it sure looks like it in turn uses an UCESB unpacker.  With the
> >> source in upexps.  That (upexps) has maintenance issues.
> >>
> >> Perhaps someone else reading this mailing list is also using unpackers
> >> from the upexps?  If so, now would be a good time to speak up, because I
> >> think we need to figure out how to have something backwards compatible
> and
> >> maintainable for you, without being 'encouraged' to do the maintenance
> >> work that other users at GSI (R3B) refuse to do...
> >>
> >> > Here, we are not sure up to which point there is a common approach to
> read
> >> > experimental data and beyond that everybody is on his own.
> >>
> >> ... while still sharing as much as possible with other users.
> >>
> >> > Many thanks and best greetings from Jena
> >> > Günter
> >>
> >> Cheers,
> >> Håkan
> >
> > --
> > subexp-daq mailing list
> > subexp-daq at lists.chalmers.se
> > https://lists.chalmers.se/mailman/listinfo/subexp-daq
> 
>