[subexp-daq] How to start a DAQ system after NURDLIB, TRLOII, DRASI, etc. were updated
Weber, Guenter Dr.
g.weber at hi-jena.gsi.de
Mon Jan 22 11:40:41 CET 2024
Dear Hans, dear Håkan,
I now recompiled R3BFUSER, but still I get the same error message when starting DRASI.
This is how the beginning how the beginning of main.cfg looks like:
log_level=verbose # info, verbose, debug, spam
CRATE("MCAL") {
GSI_VULOM(0x03000000) {
timestamp = true # needed to get timestamps in the data output
}
BARRIER
GSI_VETAR {
dactl = false
direct = false
}
BARRIER
SIS_3316(0x30000000) {
...
Do you have any idea why the dactl command is not find found? Maybe NURDLIB on GITLAB does not have the modified VETAR files yet? I have next to zero experience, so probably I am wrong. But if the last change in MODULE is from six days ago, it might not be the most recent version, right?
[cid:5f466f60-98e4-492f-9be7-7b138bb94c72]
Best greetings
Günter
________________________________
Von: subexp-daq <subexp-daq-bounces at lists.chalmers.se> im Auftrag von Håkan T Johansson <f96hajo at chalmers.se>
Gesendet: Montag, 22. Januar 2024 11:13:36
An: Discuss use of Nurdlib, TRLO II, drasi and UCESB.
Betreff: Re: [subexp-daq] How to start a DAQ system after NURDLIB, TRLOII, DRASI, etc. were updated
Dear Günter,
yes, r3bfuser depends on all of them (nurdlib, trloii, drasi), so needs a
recompile. nurdlib also needs recompile if trloii is updated.
Cheers,
Håkan
On Mon, 22 Jan 2024, Weber, Guenter Dr. wrote:
>
> Dear Hans,
>
>
> I updated the NURDLIB, but the new flag is not recognized.
>
>
> 10: config/parser.c:299: .Closed'/LynxOS/mbsusr/mbsdaq/mbsrun/rio4/2024_mcalstruck/nurdlib/cfg/default/modu
> le_log_level.cfg' }
> 5: config/keyword.c:20: .Invalid keyword "dactl".
> 5: config/keyword.c:20: .Calling abort()...
>
> Do I also need to recompile DRASI or R3BFUSER?
>
>
>
>
> Best greetings
>
> Günter
>
>
>
> ____________________________________________________________________________
> Von: subexp-daq <subexp-daq-bounces at lists.chalmers.se> im Auftrag von Hans
> Toshihide Törnqvist <hans.tornqvist at chalmers.se>
> Gesendet: Freitag, 19. Januar 2024 16:31:37
> An: Discuss use of Nurdlib, TRLO II, drasi and UCESB.
> Betreff: Re: [subexp-daq] How to start a DAQ system after NURDLIB, TRLOII,
> DRASI, etc. were updated
> Dear Günter,
>
> I have implemented the dactl switch in 'mcal_daq_merge', so in main.cfg:
>
> GSI_VETAR {
> dactl = false
> direct = false
> }
>
> I do not know how the names of the Etherbone libraries (e.g. cherry and
> enigma) relate to the driver. If the dactl file is not available, we
> just have to skip dactl accesses and not allow the direct readout path.
>
> Have a good weekend you too!
>
> Best regards,
> Hans
>
> On 2024-01-19 15:22, Weber, Guenter Dr. wrote:
> > Dear Hans,
> >
> >
> > with the help of Håkan, I managed to get the VULOM running. So, we are
> > back to starting the DRASI. There we encountered first the fact that in
> > main.cfg a BARRIER is now required between the initialization of all
> > modules (before there was only a single BARRIER command in the whole
> > main.cfg.
> >
> >
> > Now, we are again encounter the VETAR error message. See below:
> >
> >
> > 10: lwroc_hostname_util.c:108: Host 'lyserv' known as 192.168.1.1 (port:
> > 56583).
> > Thread has no error buffer yet...
> > CPUS: 1
> > delay: 1
> > 10: lwroc_hostname_util.c:108: Host 'lyserv' known as 192.168.1.1 (port:
> > 56583).
> > Thread has no error buffer yet...
> > HOST: RIO4-MCAL-1
> > Token: d6d68d7b (d6d68d7b:d6d68d7b) [/mbsusr/mbsdaq/.drasi_tokens/mcal]
> > 10: lwroc_hostname_util.c:457: Own address: 192.168.1.71/255.255.255.0
> > (eth1).
> > 10: lwroc_data_pipe.c:145: Data buffer READOUT_PIPE, size 419430400 =
> > 0x19000000, 1 consumers.
> > 10: lwroc_triva_readout.c:66: Silence TRIVA (HALT)
> > 10: lwroc_net_io.c:167: Started server on port 56583 (data port 50126).
> > client union size: 244 208 188 508 640 204 204 => 640
> > 10: lwroc_udp_awaken_hints.c:159: UDP awaken hints file:
> > /tmp/drasi.u1001/drasi.hints.u1001.RIO4-MCAL-1:56583
> > 10: lwroc_main.c:706: Log message rate limit not in effect.
> > 10: lwroc_readout.c:112: call readout_init...
> > 10: lwroc_thread_util.c:117: This is the triva control thread!
> > 10: lwroc_thread_util.c:117: This is the net io thread!
> > 10: lwroc_thread_util.c:117: This is the slow_async thread!
> > 10: lwroc_thread_util.c:117: This is the data server thread!
> > 10: lwroc_message_internal.c:472: Message client connected!
> > 10: lwroc_net_trans.c:1156: [drasi] Transport client connected (data)
> > [192.168.1.1].
> > 10: lwroc_triva_control.c:370: Setup TRIVA (DISBUS, HALT, MASTER, RESET)
> > 10: lwroc_triva_control.c:418: Minimum event time
> > ctime(5000)+1*rd(688)+3*wr(634)+fctime(1000)=8590 ns (116.414 kHz)
> > 10: lwroc_triva_state.c:1486: (Re)send ident messages...
> > 10: lwroc_triva_control.c:495: START TEST ACQ: HALT, CLEAR=RESET, MT=1
> > 9: lwroc_triva_control.c:507: TEST: GO
> > 10: lwroc_triva_control.c:725: RUN: RESET
> > 10: lwroc_triva_control.c:729: RUN: MT=14
> > 9: lwroc_triva_control.c:737: GO (1 good test triggers done) (max
> > 116.4 kHz)
> > 10: lwroc_triva_readout.c:376: Trigger 14 seen.
> > 10: config/config.c:181: Will try default cfg
> >path='/LynxOS/mbsusr/mbsdaq/mbsrun/rio4/2024_mcalstruck/nurdlib/cfg/default
> ', can be set with NURDLIB_DEF_PATH.
> > 10: config/parser.c:287: Opened
> >'/LynxOS/mbsusr/mbsdaq/mbsrun/rio4/2024_mcalstruck/nurdlib/cfg/default/glob
> al.cfg' {
> > 10: config/parser.c:299: Closed
> >'/LynxOS/mbsusr/mbsdaq/mbsrun/rio4/2024_mcalstruck/nurdlib/cfg/default/glob
> al.cfg' }
> > 8: lwroc_triva_state.c:2399: Master: deadtime: 1. Status: 0x10
> > (IN_READOUT). EC: 1
> > 10: lwroc_triva_state.c:2428: [EB lyserv] EB: Status: 0x0.
> > 8: lwroc_triva_state.c:2488: Node(s) busy in readout, waiting...
> > 10: config/parser.c:287: Opened './main.cfg' {
> > 10: config/config.c:1299: .Global log level=verbose.
> > 10: config/parser.c:287: .Opened
> >'/LynxOS/mbsusr/mbsdaq/mbsrun/rio4/2024_mcalstruck/nurdlib/cfg/default/crat
> e.cfg' {
> > 10: config/parser.c:299: .Closed
> >'/LynxOS/mbsusr/mbsdaq/mbsrun/rio4/2024_mcalstruck/nurdlib/cfg/default/crat
> e.cfg' }
> > 10: config/parser.c:287: .Opened
> >'/LynxOS/mbsusr/mbsdaq/mbsrun/rio4/2024_mcalstruck/nurdlib/cfg/default/gsi_
> vulom.cfg' {
> > 10: config/parser.c:299: .Closed
> >'/LynxOS/mbsusr/mbsdaq/mbsrun/rio4/2024_mcalstruck/nurdlib/cfg/default/gsi_
> vulom.cfg' }
> > 10: config/parser.c:287: .Opened
> >'/LynxOS/mbsusr/mbsdaq/mbsrun/rio4/2024_mcalstruck/nurdlib/cfg/default/modu
> le_log_level.cfg' {
> > 10: config/parser.c:299: .Closed
> >'/LynxOS/mbsusr/mbsdaq/mbsrun/rio4/2024_mcalstruck/nurdlib/cfg/default/modu
> le_log_level.cfg' }
> > 10: config/parser.c:287: .Opened
> >'/LynxOS/mbsusr/mbsdaq/mbsrun/rio4/2024_mcalstruck/nurdlib/cfg/default/gsi_
> vetar.cfg' {
> > 10: config/parser.c:299: .Closed
> >'/LynxOS/mbsusr/mbsdaq/mbsrun/rio4/2024_mcalstruck/nurdlib/cfg/default/gsi_
> vetar.cfg' }
> > 10: config/parser.c:287: .Opened
> >'/LynxOS/mbsusr/mbsdaq/mbsrun/rio4/2024_mcalstruck/nurdlib/cfg/default/modu
> le_log_level.cfg' {
> > 10: config/parser.c:299: .Closed
> >'/LynxOS/mbsusr/mbsdaq/mbsrun/rio4/2024_mcalstruck/nurdlib/cfg/default/modu
> le_log_level.cfg' }
> > 10: config/parser.c:287: .Opened
> >'/LynxOS/mbsusr/mbsdaq/mbsrun/rio4/2024_mcalstruck/nurdlib/cfg/default/sis_
> 3316.cfg' {
> > 10: config/parser.c:299: .Closed
> >'/LynxOS/mbsusr/mbsdaq/mbsrun/rio4/2024_mcalstruck/nurdlib/cfg/default/sis_
> 3316.cfg' }
> > 10: config/parser.c:287: .Opened
> >'/LynxOS/mbsusr/mbsdaq/mbsrun/rio4/2024_mcalstruck/nurdlib/cfg/default/modu
> le_log_level.cfg' {
> > 10: config/parser.c:299: .Closed
> >'/LynxOS/mbsusr/mbsdaq/mbsrun/rio4/2024_mcalstruck/nurdlib/cfg/default/modu
> le_log_level.cfg' }
> > 10: config/parser.c:287: .Opened
> >'/LynxOS/mbsusr/mbsdaq/mbsrun/rio4/2024_mcalstruck/nurdlib/cfg/default/sis_
> 3316.cfg' {
> > 10: config/parser.c:299: .Closed
> >'/LynxOS/mbsusr/mbsdaq/mbsrun/rio4/2024_mcalstruck/nurdlib/cfg/default/sis_
> 3316.cfg' }
> > 10: config/parser.c:287: .Opened
> >'/LynxOS/mbsusr/mbsdaq/mbsrun/rio4/2024_mcalstruck/nurdlib/cfg/default/modu
> le_log_level.cfg' {
> > 10: config/parser.c:299: .Closed
> >'/LynxOS/mbsusr/mbsdaq/mbsrun/rio4/2024_mcalstruck/nurdlib/cfg/default/modu
> le_log_level.cfg' }
> > 10: config/parser.c:299: Closed './main.cfg' }
> > 10: crate/crate.c:347: crate_create {
> > 10: crate/crate.c:673: crate_create(MCAL) }
> > 10: crate/crate.c:899: crate_init(MCAL) {
> > 10: crate/crate.c:923: .Slow-init module[0]=GSI_VULOM.
> > LOG: TRLO: MD5SUM: 0x1409285e (CT: 63bb1d44 = 2023-01-08 19:45:08 UTC)
> > 10: crate/crate.c:923: .Slow-init module[1]=GSI_VETAR.
> > 5: module/gsi_etherbone/gsi_etherbone.c:431: ..Gsi Etherbone
> > /sys/class/vetar/vetar0/dactl: Failed dactl fopen: No such file or
> > directory.
> > 5: module/gsi_etherbone/gsi_etherbone.c:431: ..Calling abort()...
> >
> > We are sure that the right NURDLIB path is used and also DRASI is
> > started in the correct folder.
> >
> >
> > What we found in the enironment variables are two lines, where 'white
> > rabbit' is mentioned. As I understand, this is connected to the VETAR2
> > module.
> >
> >
> >LD_LIBRARY_PATH=/mbs/driv/white_rabbit/cherry/RIO4_Linux_OSV_2_6_33_SuHa/li
> b:/mbsusr/mbsdaq/mbsrun/whiterabbit/enigma/RIO4_Linux_OSV_2_6_33_SuHa/lib
> >PATH=/mbs/driv/white_rabbit/cherry/RIO4_Linux_OSV_2_6_33_SuHa/bin:/bin:/bin
> /ces:/usr/bin:/mbsusr/mbsdaq/bin:/usr/local/bin:/usr/bin/X11:/etc:/usr/etc
> :.:/mbsusr/mbsdaq/tools
> >
> > Maybe, it is necessary to update something there?
> >
> >
> > Anyway, if you have now a switch in NURDLIB to deal with old software
> > for the VETAR, then we should proceed with removing NURDLIB and getting
> > the newer version, right?
> >
> >
> >
> >
> > Many thanks and have a nice weekend!
> >
> >
> >
> >
> > Best greetings
> >
> > Günter
> >
> >
> >
> >
> > ------------------------------------------------------------------------
> > *Von:* subexp-daq <subexp-daq-bounces at lists.chalmers.se> im Auftrag von
> > Hans Toshihide Törnqvist <hans.tornqvist at chalmers.se>
> > *Gesendet:* Donnerstag, 18. Januar 2024 13:28:36
> > *An:* Discuss use of Nurdlib, TRLO II, drasi and UCESB.; Weber, Guenter
> Dr.
> > *Betreff:* Re: [subexp-daq] How to start a DAQ system after NURDLIB,
> > TRLOII, DRASI, etc. were updated
> > Dear Günter,
> >
> > It looks like either the Vetar driver failed, or the fw/driver is old
> > and nurdlib expects a file that didn't exist a while back. If it's the
> > latter I can add a switch to ignore this file, it's not critical for
> > readout and is used to enter a faster readout mode.
> >
> > Which version of nurdlib is this? The files and line numbers don't match
> > with what I see in the source code, and the last time gsi_etherbone.c
> > changed was early last year.
> >
> > The message "Could not open so-and-so" should be for the device, e.g.
> > /dev/pcie_wb0. This is used for the readout.
> >
> > The dactl file should fail with "Failed dactl fopen".
> >
> > To summarise, I would suggest to make sure that the correct nurdlib and
> > r3bfuser are executed by the DAQ scripts. Meanwhile I will prepare a
> > switch to ignore the dactl file which could anyhow be useful for old
> > configurations, and to clean up the Etherbone error messages.
> >
> > Best regards,
> > Hans
> >
> > On 2024-01-18 11:29, Weber, Guenter Dr. wrote:
> >> Dear friends,
> >>
> >>
> >> I now started the DAQ and it looks like there is a problem with the
> >> VETAR2 module. Attached please find the output once the DAQ is started
> >> on the RIO4.
> >>
> >>
> >> Here ist the error message:
> >>
> >>
> >> 11: a:1: .Slow module[1]=GSI_VETAR. (crate/crate.c:866)
> >> 11: a:1: .Gsi Etherbone init_slow {
> >> (module/gsi_etherbone/gsi_etherbone.c:110)
> >> 5: a:1: ..Gsi Etherbone Could not open /sys/class/vetar/vetar0/dactl: No
> >> such file or directory.
> >> (module/gsi_etherbone/gsi_etherbone.c:120)
> >> 5: a:1: ..Calling exit(EXIT_FAILURE)...
> >> (module/gsi_etherbone/gsi_etherbone.c:120)
> >>
> >> How to proceed?
> >>
> >>
> >>
> >> Many thanks and best greetings
> >> Günter
> >>
> >>
> >>
> >>
> >> ------------------------------------------------------------------------
> >> *Von:* subexp-daq <subexp-daq-bounces at lists.chalmers.se> im Auftrag von
> >> Håkan T Johansson <f96hajo at chalmers.se>
> >> *Gesendet:* Dienstag, 16. Januar 2024 19:20:54
> >> *An:* Discuss use of Nurdlib, TRLO II, drasi and UCESB.
> >> *Betreff:* Re: [subexp-daq] How to start a DAQ system after NURDLIB,
> >> TRLOII, DRASI, etc. were updated
> >>
> >> On Tue, 16 Jan 2024, Weber, Guenter Dr. wrote:
> >>
> >>>
> >>> Dear friends,
> >>>
> >>>
> >>> I am just trying to figure out how all the various scripts/commands work
> >>> together for setting up our DAQ system. Maybe, you can help to clearify
> some
> >>> things.
> >>>
> >>>
> >>>
> >>> - master.bash
> >>
> >> This would be intended to run on the readout board (RIO). The other
> >> scripts below is for the PC.
> >>
> >>>
> >>> source ../env/env.sh
> >>> ./trloii_setup.sh
> >>> # gdb --args \
> >>> ../r3bfuser/build_cc_ppc-linux_4.2.2_debug_drasi/m_read_meb.drasi \
> >>> --label=MCAL1 \
> >>> --triva=master,fctime=10,ctime=50 \
> >>> --log-no-rate-limit \
> >>> --server=drasi,dest=lyserv \
> >>> --buf=size=400Mi \
> >>> --max-ev-size=0x1000000 \
> >>> --subev=crate=1,type=20,subtype=2,control=9,procid=1 \
> >>> --eb=lyserv \
> >>> "$@"
> >>>
> >>> In the first line some environment variables are set. The second line
> tells
> >>> the VULOM4 how to operate (i. e. selection from modes of operation
> defined
> >>> in vulom.trlo). Then DRASI is started with a bunch of parameters. Is
> there
> >>> somewhere an explanation of the purpose of all these settings?
> >>
> >> The drasi command line options should be described here:
> >>
> >> http://fy.chalmers.se/~f96hajo/drasi/doc/drasi_cmdline.html
> > <http://fy.chalmers.se/~f96hajo/drasi/doc/drasi_cmdline.html>
> >> <http://fy.chalmers.se/~f96hajo/drasi/doc/drasi_cmdline.html
> > <http://fy.chalmers.se/~f96hajo/drasi/doc/drasi_cmdline.html>>
> >>
> >> (Note: 'lyserv' is the name of your PC, at least as seen from the RIO.)
> >>
> >> The TRLO II setup file (vulom.trlo) is worse in terms of documentation.
> >> The format as such is hopefully rather straightforward. A description is
> >> here: http://fy.chalmers.se/~f96hajo/trloii/man5/trloconf.5.html
> > <http://fy.chalmers.se/~f96hajo/trloii/man5/trloconf.5.html>
> >> <http://fy.chalmers.se/~f96hajo/trloii/man5/trloconf.5.html
> > <http://fy.chalmers.se/~f96hajo/trloii/man5/trloconf.5.html>> .
> >> The actual meaning is more complex, the TRLO II description can be found
> >> here:
> >>
> >> http://fy.chalmers.se/~f96hajo/trloii/vulom4_trlo/descr_defs_frame.html
> > <http://fy.chalmers.se/~f96hajo/trloii/vulom4_trlo/descr_defs_frame.html>
> >> <http://fy.chalmers.se/~f96hajo/trloii/vulom4_trlo/descr_defs_frame.html
> > <http://fy.chalmers.se/~f96hajo/trloii/vulom4_trlo/descr_defs_frame.html>>
> >>
> >> (left pane), which just is a colourised version of this:
> >> http://fy.chalmers.se/~f96hajo/trloii/description.txt
> > <http://fy.chalmers.se/~f96hajo/trloii/description.txt>
> >> <http://fy.chalmers.se/~f96hajo/trloii/description.txt
> > <http://fy.chalmers.se/~f96hajo/trloii/description.txt>>
> >>
> >> It does not describe the .trlo file as such, but what the gateware tries
> >> to achieve.
> >>
> >>> And what is in the last line? I never came accross "$@" before.
> >>
> >> That would be all the (non-shifted, i.e. not removed) options given to
> the
> >> script itself, i.e. master.bash.
> >>
> >>> - eb.bash
> >>>
> >>> ../drasi/bin/lwrocmerge \
> >>>
> >>> --label=MCAL_EB \
> >>> --merge-mode=event \
> >>> --server=trans,flush=1 \
> >>> --server=stream,flush=1 \
> >>> --buf=size=1600Mi \
> >>> --max-ev-size=20Mi \
> >>> --eb-master=rio4-mcal-1 \
> >>> --file-writer \
> >>> --drasi=rio4-mcal-1
> >>>
> >>> What is the purpose of this command?
> >>
> >> This runs an 'event builder' on the PC. Not strictly needed, but this
> >> way, the RIO only ever needs to send data once, to this event-builder
> >> (EB). File writing, and streaming of on-line data is then handled by
> this
> >> process. It is cheaper to have better network cards and so on in a plain
> >> PC.
> >>
> >>> Why are values for buffer size and max
> >>> event size different then in the command before?
> >>
> >> The nomenclaturure is unfortunately a bit convoluted below with ucesb
> >> regards 'buffer size'. The --buf in the two commands above give the size
> >> of the memory used to buffer data inside the process, and depends on how
> >> much memory each machine has. The --max-ev-size tells how large each
> .lmd
> >> event can be. The --max-ev-size configured in the event builder needs to
> >> be as large as the sum of all event sources it has. I.e. should be able
> >> to be the same as in your single source above.
> >>
> >> In the readout itself, any event that is read out cannot be larger than
> >> this. If it is, then the DAQ will report a fatal failure.
> >>
> >>> - fanout.bash
> >>>
> >>> #!/bin/bash
> >>> while :
> >>> do
> >>> ~/mbsrun/rio4/mcalstruck/ucesb/empty/empty \
> >>> stream://localhost \
> >>> --server=stream:8001,bufsize=10Mi,flush=1,dataport:7001
> >>> sleep 5
> >>> done
> >>>
> >>> This looks like setting up a connection point to the outside world,
> right?
> >>
> >> Yes. It accepts connections and will serve them data for online
> analysis.
> >>
> >> stream://localhost tells it where to read data (here the PC itself where
> >> it is running). Using the 'stream' protocol, which esentially is just
> raw
> >> LMD events. (As is the 'transport protocol'.) That will be served on
> the
> >> default port used for the stream server set up by the EB above.
> >>
> >> --server is then the server that accepts connections, and serves data on
> >> another port (8001).
> >>
> >>> And we find the third (random?) value for a buffer size.
> >>
> >> That bufsize is actually the LMD buffer size used by the stream server.
> >> The corresponding value in the readout and EB is set implicitly from the
> >> maximum event size...
> >>
> >>> - rate.bash
> >>>
> >>> ../drasi/bin/lwrocmon --rate rio4-mcal-1
> >>>
> >>> Ok, this I understand. We can look the DAQ over the shoulder and see
> what
> >>> the thing is doing in terms of number of trigger events, transported
> data,
> >>> etc.
> >>
> >>> - log.bash
> >>>
> >>> ../drasi/bin/lwrocmon --log rio4-mcal-1 localhost
> >>>
> >>> This is also easy to understand.
> >>
> >> :-)
> >>
> >> Then also try to do (on the PC)
> >>
> >> drasi/bin/lwrocmon --tree rio4-mcal-1 localhost
> >>
> >> which would show a 'tree-view' monitoring of the running system.
> >>
> >>
> >> Cheers,
> >> Håkan
> >>
> > --
> > subexp-daq mailing list
> > subexp-daq at lists.chalmers.se
> > https://lists.chalmers.se/mailman/listinfo/subexp-daq
> > <https://lists.chalmers.se/mailman/listinfo/subexp-daq>
> >
> --
> subexp-daq mailing list
> subexp-daq at lists.chalmers.se
> https://lists.chalmers.se/mailman/listinfo/subexp-daq
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.chalmers.se/pipermail/subexp-daq/attachments/20240122/6dfd7f35/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pastedImage.png
Type: image/png
Size: 168174 bytes
Desc: pastedImage.png
URL: <http://lists.chalmers.se/pipermail/subexp-daq/attachments/20240122/6dfd7f35/attachment-0001.png>
More information about the subexp-daq
mailing list