[subexp-daq] How to start a DAQ system after NURDLIB, TRLOII, DRASI, etc. were updated
Håkan T Johansson
f96hajo at chalmers.se
Fri Jan 19 11:29:14 CET 2024
Dear Günter,
yes, the gateware running on the VULOM is an older version, so not the one
that trloctrl etc. has now been compiled with. Could you use vulomflash
to --prog the new firmware e.g. to range 2, and then restart from that.
See the instructions at the end of the attached file.
Cheers,
Håkan
On Fri, 19 Jan 2024, Weber, Guenter Dr. wrote:
>
> P.S.
>
>
> Maybe this information helps:
>
>
> RIO4-MCAL-1 mbsdaq > vulomflash --addr=3 --readprogs
> VULOM base address: 0x03000000
> hwmap_mapvme.c:398: LOG: Virtual address for VULOM/TRIDI @ VME 0x03000000 is
> 0x3005e000.
> Performing command 'readprogs'...
> =========================================================================
> Rng 0: TRLO II ver/vulom4b_trlo 2018-10-07 21:56:55 ( 9.486ns)
> 426cb99c
> Rng 1: TRLO II ver/vulom4b_trlo 2018-10-07 21:56:55 ( 9.486ns)
> 426cb99c
> Rng 2: N/A
> Rng 3: N/A
> Rng 4: N/A
> Rng 5: N/A
> Rng 6: N/A
> Rng 7: N/A
> =========================================================================
> Released vme ptr.
>
>
> Best greetings
>
> Günter
>
>
>
> ____________________________________________________________________________
> Von: subexp-daq <subexp-daq-bounces at lists.chalmers.se> im Auftrag von Weber,
> Guenter Dr. <g.weber at hi-jena.gsi.de>
> Gesendet: Freitag, 19. Januar 2024 10:16:03
> An: Discuss use of Nurdlib, TRLO II, drasi and UCESB.
> Betreff: Re: [subexp-daq] How to start a DAQ system after NURDLIB, TRLOII,
> DRASI, etc. were updated
>
> Sorry, please ignore my last e-mail.
>
>
> Now the automatic setting of the folder works.
>
>
> But there is a real problem after executing of the VULOM command.
>
>
> hwmap_mapvme.c:398: LOG: Virtual address for TRLO II @ VME 0x03000000 is
> 0x3005e000.
> LOG: TRLO: MD5SUM: 0x426cb99c (CT: 5bba6507 = 2018-10-07 19:56:55 UTC)
> WARNING: Known firmware (alias): 0x6e4ba1a9.
> WARNING: Known firmware (alias): 0x1409285e.
> WARNING: Known firmware (alias): 0xa73c5093.
> WARNING: Known firmware (alias): 0x6e4ba1a9.
> WARNING: Known firmware (alias): 0x1409285e.
> FATAL: TRLO firmware wrong: 0x426cb99c, expected 0x6e4ba1a9 or alias.
>
>
>
> Best greetings
>
> Günter
>
>
>
> ____________________________________________________________________________
> Von: subexp-daq <subexp-daq-bounces at lists.chalmers.se> im Auftrag von Weber,
> Guenter Dr. <g.weber at hi-jena.gsi.de>
> Gesendet: Freitag, 19. Januar 2024 09:58:21
> An: Discuss use of Nurdlib, TRLO II, drasi and UCESB.
> Betreff: Re: [subexp-daq] How to start a DAQ system after NURDLIB, TRLOII,
> DRASI, etc. were updated
>
> Dear Håkan, dear Hans,
>
>
> it looks like the FW number that is identified is the wrong one.
>
>
> vulom_fw=`cat ${TRLOII_PATH}/fw/vulom4b_trlo/trlo_defs.h 2>&1 | grep
> MD5SUM_STAMP | sed 's/.*0x//'`
> exportVULOM4_CTRL=$trloiipath/trloctrl/fw_${VULOM4_FW}_trlo/bin_${machine}_${vers
> ion}/trlo_ctrl
>
> On my system this results in the following path:
>
>
> VULOM4_CTRL=/LynxOS/mbsusr/mbsdaq/mbsrun/rio4/2024_mcalstruck/trloii/trloct
> rl/fw_d96ffc88_trlo/bin_ppc-linux_4.2.2/trlo_ctrl
>
>
> But it is the wrong one.
>
>
> $VULOM4_CTRL --addr=$addr --clear-setup --config=vulom.trlo standalone
> module_trigger
>
> ./trloii_setup.sh: line 12:/LynxOS/mbsusr/mbsdaq/mbsrun/rio4/2024_mcalstruck/trloii/trloctrl/fw_d96ffc
> 88_trlo/bin_ppc-linux_4.2.2/trlo_ctrl: No such file or directory
>
>
> The right one would be:
>
>
> /LynxOS/mbsusr/mbsdaq/mbsrun/rio4/2024_mcalstruck/trloii/trloctrl/fw_140928
> 5e_trlo/bin_ppc-linux_4.2.2/trlo_ctrl
>
>
> In the TRLO/TRLOCTRL folder there are also other firmware numbers (which
> redirect to 1409285e, as I understand), but d96ffc88 is not among them.
>
> Do you guys have an idea what the problem is?
>
> I know that this is a side issue and I could hard code the right path, but I
> would like to make the thing as convenient as possible before I hand it over
> to the students 😊
>
>
>
> Best greetings
> Günter
>
> ____________________________________________________________________________
> Von: subexp-daq <subexp-daq-bounces at lists.chalmers.se> im Auftrag von Håkan
> T Johansson <f96hajo at chalmers.se>
> Gesendet: Donnerstag, 18. Januar 2024 22:39:31
> An: Discuss use of Nurdlib, TRLO II, drasi and UCESB.
> Betreff: Re: [subexp-daq] How to start a DAQ system after NURDLIB, TRLOII,
> DRASI, etc. were updated
>
> On Thu, 18 Jan 2024, Weber, Guenter Dr. wrote:
>
> >
> > Dear Hans,
> >
> >
> > there was some redirect burried in the script which executes scripts which
> > execute some more scripts ... and that made me start the DAQ in the old
> > folder, not the new one. Sorry for that.
> >
> >
> > I now came across the following line in the script which starts the VULOM:
> >
> >
> > $VULOM4_CTRL --addr=$addr --clear-setup --config=vulom.trlo standalone
> > module_trigger
> >
> >
> > Here I need to know the firmware number to define the path VULOM4_CTRL,
> > right? Is there a way to do this automatically? Somehow this works already
> > in the makefile of NURDLIB.
>
> It looks like this is done with this line in the nurdlib Makefile:
>
> VULOM4_FW:=$(shell cat $(TRLOII_PATH)/fw/vulom4b_trlo/trlo_defs.h 2>&1 |
> grep MD5SUM_STAMP | sed 's/.*0x//')
>
> such that something like
>
> VULOM4_FW=`cat $(TRLOII_PATH)/fw/vulom4b_trlo/trlo_defs.h 2>&1 | grep
> MD5SUM_STAMP | sed 's/.*0x//'`
>
> might work in a shell script, if TRLOII_PATH is set, and the fw/ directory
> exists and has the current gatewares.
>
> 'Funfact': This general mess with the TRLO II version numbers is of my
> creation, however a proper fix has so far required too much effort.
> Basically, the gateware would have to give info on the offsets and sizes
> of various controllable items (fairly easy), and the trloctrl program and
> functions would need to use that (much more complex - that it at all works
> is thanks to generation of many structures from that trlo_defs.h header,
> where some of them also intermix with the .trlo lexer/parser).
>
> > I would like to avoid to hard code this into the environment settings if
> > there is a way to figure it out on the fly.
>
> Cheers,
> Håkan
>
>
>
> >
> >
> >
> > Thanks so much!
> >
> >
> >
> > Best greetings
> >
> > Günter
> >
> >
> >
> >
> >
> >
> >___________________________________________________________________________
> _
> > Von: subexp-daq <subexp-daq-bounces at lists.chalmers.se> im Auftrag von Hans
> > Toshihide Törnqvist <hans.tornqvist at chalmers.se>
> > Gesendet: Donnerstag, 18. Januar 2024 13:28:36
> > An: Discuss use of Nurdlib, TRLO II, drasi and UCESB.; Weber, Guenter Dr.
> > Betreff: Re: [subexp-daq] How to start a DAQ system after NURDLIB, TRLOII,
> > DRASI, etc. were updated
> > Dear Günter,
> >
> > It looks like either the Vetar driver failed, or the fw/driver is old
> > and nurdlib expects a file that didn't exist a while back. If it's the
> > latter I can add a switch to ignore this file, it's not critical for
> > readout and is used to enter a faster readout mode.
> >
> > Which version of nurdlib is this? The files and line numbers don't match
> > with what I see in the source code, and the last time gsi_etherbone.c
> > changed was early last year.
> >
> > The message "Could not open so-and-so" should be for the device, e.g.
> > /dev/pcie_wb0. This is used for the readout.
> >
> > The dactl file should fail with "Failed dactl fopen".
> >
> > To summarise, I would suggest to make sure that the correct nurdlib and
> > r3bfuser are executed by the DAQ scripts. Meanwhile I will prepare a
> > switch to ignore the dactl file which could anyhow be useful for old
> > configurations, and to clean up the Etherbone error messages.
> >
> > Best regards,
> > Hans
> >
> > On 2024-01-18 11:29, Weber, Guenter Dr. wrote:
> > > Dear friends,
> > >
> > >
> > > I now started the DAQ and it looks like there is a problem with the
> > > VETAR2 module. Attached please find the output once the DAQ is started
> > > on the RIO4.
> > >
> > >
> > > Here ist the error message:
> > >
> > >
> > > 11: a:1: .Slow module[1]=GSI_VETAR. (crate/crate.c:866)
> > > 11: a:1: .Gsi Etherbone init_slow {
> > > (module/gsi_etherbone/gsi_etherbone.c:110)
> > > 5: a:1: ..Gsi Etherbone Could not open /sys/class/vetar/vetar0/dactl: No
> > > such file or directory.
> > > (module/gsi_etherbone/gsi_etherbone.c:120)
> > > 5: a:1: ..Calling exit(EXIT_FAILURE)...
> > > (module/gsi_etherbone/gsi_etherbone.c:120)
> > >
> > > How to proceed?
> > >
> > >
> > >
> > > Many thanks and best greetings
> > > Günter
> > >
> > >
> > >
> > >
> > > ------------------------------------------------------------------------
> > > *Von:* subexp-daq <subexp-daq-bounces at lists.chalmers.se> im Auftrag von
> > > Håkan T Johansson <f96hajo at chalmers.se>
> > > *Gesendet:* Dienstag, 16. Januar 2024 19:20:54
> > > *An:* Discuss use of Nurdlib, TRLO II, drasi and UCESB.
> > > *Betreff:* Re: [subexp-daq] How to start a DAQ system after NURDLIB,
> > > TRLOII, DRASI, etc. were updated
> > >
> > > On Tue, 16 Jan 2024, Weber, Guenter Dr. wrote:
> > >
> > >>
> > >> Dear friends,
> > >>
> > >>
> > >> I am just trying to figure out how all the various scripts/commands
> work
> > >> together for setting up our DAQ system. Maybe, you can help to clearify
> > some
> > >> things.
> > >>
> > >>
> > >>
> > >> - master.bash
> > >
> > > This would be intended to run on the readout board (RIO). The other
> > > scripts below is for the PC.
> > >
> > >>
> > >> source ../env/env.sh
> > >> ./trloii_setup.sh
> > >> # gdb --args \
> > >> ../r3bfuser/build_cc_ppc-linux_4.2.2_debug_drasi/m_read_meb.drasi \
> > >> --label=MCAL1 \
> > >> --triva=master,fctime=10,ctime=50 \
> > >> --log-no-rate-limit \
> > >> --server=drasi,dest=lyserv \
> > >> --buf=size=400Mi \
> > >> --max-ev-size=0x1000000 \
> > >> --subev=crate=1,type=20,subtype=2,control=9,procid=1 \
> > >> --eb=lyserv \
> > >> "$@"
> > >>
> > >> In the first line some environment variables are set. The second line
> > tells
> > >> the VULOM4 how to operate (i. e. selection from modes of operation
> > defined
> > >> in vulom.trlo). Then DRASI is started with a bunch of parameters. Is
> > there
> > >> somewhere an explanation of the purpose of all these settings?
> > >
> > > The drasi command line options should be described here:
> > >
> > > http://fy.chalmers.se/~f96hajo/drasi/doc/drasi_cmdline.html
> > > <http://fy.chalmers.se/~f96hajo/drasi/doc/drasi_cmdline.html>
> > >
> > > (Note: 'lyserv' is the name of your PC, at least as seen from the RIO.)
> > >
> > > The TRLO II setup file (vulom.trlo) is worse in terms of documentation.
> > > The format as such is hopefully rather straightforward. A description
> is
> > > here: http://fy.chalmers.se/~f96hajo/trloii/man5/trloconf.5.html
> > > <http://fy.chalmers.se/~f96hajo/trloii/man5/trloconf.5.html> .
> > > The actual meaning is more complex, the TRLO II description can be found
> > > here:
> > >
> > > http://fy.chalmers.se/~f96hajo/trloii/vulom4_trlo/descr_defs_frame.html
> > >
> <http://fy.chalmers.se/~f96hajo/trloii/vulom4_trlo/descr_defs_frame.html>
> > >
> > > (left pane), which just is a colourised version of this:
> > > http://fy.chalmers.se/~f96hajo/trloii/description.txt
> > > <http://fy.chalmers.se/~f96hajo/trloii/description.txt>
> > >
> > > It does not describe the .trlo file as such, but what the gateware tries
> > > to achieve.
> > >
> > >> And what is in the last line? I never came accross "$@" before.
> > >
> > > That would be all the (non-shifted, i.e. not removed) options given to
> the
> > > script itself, i.e. master.bash.
> > >
> > >> - eb.bash
> > >>
> > >> ../drasi/bin/lwrocmerge \
> > >>
> > >> --label=MCAL_EB \
> > >> --merge-mode=event \
> > >> --server=trans,flush=1 \
> > >> --server=stream,flush=1 \
> > >> --buf=size=1600Mi \
> > >> --max-ev-size=20Mi \
> > >> --eb-master=rio4-mcal-1 \
> > >> --file-writer \
> > >> --drasi=rio4-mcal-1
> > >>
> > >> What is the purpose of this command?
> > >
> > > This runs an 'event builder' on the PC. Not strictly needed, but this
> > > way, the RIO only ever needs to send data once, to this event-builder
> > > (EB). File writing, and streaming of on-line data is then handled by
> this
> > > process. It is cheaper to have better network cards and so on in a
> plain
> > > PC.
> > >
> > >> Why are values for buffer size and max
> > >> event size different then in the command before?
> > >
> > > The nomenclaturure is unfortunately a bit convoluted below with ucesb
> > > regards 'buffer size'. The --buf in the two commands above give the
> size
> > > of the memory used to buffer data inside the process, and depends on how
> > > much memory each machine has. The --max-ev-size tells how large each
> .lmd
> > > event can be. The --max-ev-size configured in the event builder needs
> to
> > > be as large as the sum of all event sources it has. I.e. should be able
> > > to be the same as in your single source above.
> > >
> > > In the readout itself, any event that is read out cannot be larger than
> > > this. If it is, then the DAQ will report a fatal failure.
> > >
> > >> - fanout.bash
> > >>
> > >> #!/bin/bash
> > >> while :
> > >> do
> > >> ~/mbsrun/rio4/mcalstruck/ucesb/empty/empty \
> > >> stream://localhost \
> > >> --server=stream:8001,bufsize=10Mi,flush=1,dataport:7001
> > >> sleep 5
> > >> done
> > >>
> > >> This looks like setting up a connection point to the outside world,
> > right?
> > >
> > > Yes. It accepts connections and will serve them data for online
> analysis.
> > >
> > > stream://localhost tells it where to read data (here the PC itself
> where
> > > it is running). Using the 'stream' protocol, which esentially is just
> raw
> > > LMD events. (As is the 'transport protocol'.) That will be served on
> the
> > > default port used for the stream server set up by the EB above.
> > >
> > > --server is then the server that accepts connections, and serves data on
> > > another port (8001).
> > >
> > >> And we find the third (random?) value for a buffer size.
> > >
> > > That bufsize is actually the LMD buffer size used by the stream server.
> > > The corresponding value in the readout and EB is set implicitly from the
> > > maximum event size...
> > >
> > >> - rate.bash
> > >>
> > >> ../drasi/bin/lwrocmon --rate rio4-mcal-1
> > >>
> > >> Ok, this I understand. We can look the DAQ over the shoulder and see
> what
> > >> the thing is doing in terms of number of trigger events, transported
> > data,
> > >> etc.
> > >
> > >> - log.bash
> > >>
> > >> ../drasi/bin/lwrocmon --log rio4-mcal-1 localhost
> > >>
> > >> This is also easy to understand.
> > >
> > > :-)
> > >
> > > Then also try to do (on the PC)
> > >
> > > drasi/bin/lwrocmon --tree rio4-mcal-1 localhost
> > >
> > > which would show a 'tree-view' monitoring of the running system.
> > >
> > >
> > > Cheers,
> > > Håkan
> > >
> > --
> > subexp-daq mailing list
> > subexp-daq at lists.chalmers.se
> > https://lists.chalmers.se/mailman/listinfo/subexp-daq
> >
> >
>
>
-------------- next part --------------
######################################################################
MYDAQ=/path/to/daq
mkdir -p $MYDAQ
cd $MYDAQ
git clone git at gitlab.com:chalmers-subexp/nurdlib.git
git clone git at gitlab.com:chalmers-subexp/trloii.git
git clone git at gitlab.com:chalmers-subexp/r3bfuser.git
git clone git at git.chalmers.se:expsubphys/drasi.git
git clone git at git.chalmers.se:expsubphys/ucesb.git
######################################################################
# Download TRLO II gatewares and unpack (on PC):
cd $MYDAQ/trloii
wget http://fy.chalmers.se/~f96hajo/trloii/trloii_firmwares_f13d071c.tar.gz
tar -zxvf trloii_firmwares_f13d071c.tar.gz
######################################################################
cd $MYDAQ/trloii
# Workarounds (in case of compile issues below):
# In trloii/Makefile: Partially comment out line 3:
proglinks # trigalign_dir
# In trloii/trloctrl/trlolib/src/trlo_setup_parser.y: Comment out line 60:
/* %define parse.error verbose */
######################################################################
# Build (PC, then VME board).
cd $MYDAQ/trloii
make
cd trloctrl
./find_firmwares.pl
make fw_1409285e_trlo_build # choose the vulom4b trlo
##
cd $MYDAQ/drasi
make
##
cd $MYDAQ/nurdlib
make
##
cd $MYDAQ/r3bfuser
make drasi
######################################################################
## Build (PC only).
cd $MYDAQ/ucesb
make empty
######################################################################
# Testing and upgrading TRLO II gateware (on VME board):
ADDR=XX # Address selectors on VULOM4(b) board
cd $MYDAQ/trloii
# List gatewares in the 8 flash regions:
bin/vulomflash --addr=$ADDR --readprogs
# See which gateware is currently running (Look for output on line
# VOLUM+0 => 0xZZZZ1f20, ZZZZ is four first hex digits of identifier.)
bin/vulomflash --addr=$ADDR --read
# Restart FPGA with gateware from region N:
bin/vulomflash --addr=$ADDR --restart=N
# Write a (new) gateware image to region N, N _not_ 0:
bin/vulomflash --addr=$ADDR --prog=N fw/vulom4b_trlo/vlogic_4b.rbt
# Then check that it works:
bin/vulomflash --addr=$ADDR --restart=N
bin/vulomflash --addr=$ADDR --read
# Also use with a DAQ before writing to region 0, which is loaded on
# cold start. If region 0 image is bad, it may jam the VME bus and
# prevent re-programming. (--restart=N can be used after cold start.)
##
# Just using the trlo_ctrl to talk to the module.
# Without any argument, shall respond with two lines, second
# contain identifier and gateware synthesis time.
trloctrl/fw_1409285e_trlo/bin_ppc-.../trlo_ctrl --addr=$ADDR
######################################################################
More information about the subexp-daq
mailing list