[subexp-daq] Report of a possible bug of the CAEN_V560 module

[Håkan T Johansson]

Dear Günter,

I took the files you provided and for comparison put them in a branch 
'old_caen_v560'.

git diff origin/old_caen_v560..origin/master

however does not show anything which is suspicious to me.  Perhaps Hans 
can spot something.

Otherwise, the only idea I can come up with is to continue to bisect the 
code inside slow init.

However, before that, I would suggest to add

  fflush(stdout); sleep(1);

after each printf statement, such that one can be quite sure that the 
printout is not eaten when the RIO crash happens.  I.e. that it actually 
had gotten further than shown by the prints.

Best regards,
Håkan




On Tue, 20 Feb 2024, Weber, Guenter Dr. wrote:

> 
> Dear friends,
> 
> 
> I now had a look at the system where the V560 was running. It was also setup
> by Bastian. And there the code for the V560 module is slightly different
> from the one included in the NURDLIB branch that I am using on the test
> system.
> 
> 
> Maybe you can have a look at it.
> 
> 
> I also could push the complete NURDLIB from this system, if this helps.
> 
> 
> 
> 
> Best greetings
> 
> Günter
> 
> 
> 
> 
> ____________________________________________________________________________
> Von: subexp-daq <subexp-daq-bounces at lists.chalmers.se> im Auftrag von Weber,
> Guenter Dr. <g.weber at hi-jena.gsi.de>
> Gesendet: Dienstag, 20. Februar 2024 10:58:27
> An: Discuss use of Nurdlib, TRLO II, drasi and UCESB.
> Betreff: [subexp-daq] Report of a possible bug of the CAEN_V560 module  
> 
> Dear friends,
> 
> 
> I now grabbed a V560 module that was working fine in another DAQ system and
> put it into our test system.
> 
> 
> The main.cfg looks like this:
> 
> 
> log_level=spam # info, verbose, debug, spam
> 
> CRATE("MCAL") {
>     GSI_VULOM(0x03000000) {
>         timestamp = true # needed to get timestamps in the data output
>     #   ecl=0..15
>     }
>     BARRIER
>     CAEN_V560(0x333333300) {
>         use_veto = true
>     }  
> #   CAEN_V767A(0x03100000) {
> #   }
> }
> 
> Starting the DAQ now results in a freeze of the RIO4. A reset of the crate
> is necessary to talk to it again.
> 
> 
> The problem occurs in the first slow init of the V560 module. To find the
> exact line, I added some output to CRATE.C:
> 
> 
> 10: crate/crate.c:923: .Slow-init module[0]=GSI_VULOM.
> before push_log_level(module)
> before a_crate->module_init_id = module->id
> before module->props->init_slow(a_crate, module)
> LOG: TRLO: MD5SUM: 0x1409285e (CT: 63bb1d44 = 2023-01-08 19:45:08 UTC)
> before module_init_id_mark(a_crate, module)
> before pop_log_level(module)
> 10: crate/crate.c:923: .Slow-init module[1]=CAEN_V560.
> before push_log_level(module)
> before a_crate->module_init_id = module->id
> before module->props->init_slow(a_crate, module)
> 
> 
> The CRATE.C code now looks like this:
> 
> 
>     TAILQ_FOREACH(module, &a_crate->module_list, next) {
>         if (NULL == module->props) {
>             continue;
>         }
>         LOGF(info)(LOGL, "Slow-init module[%u]=%s.", module->id,
>             keyword_get_string(module->type));
>         printf("before push_log_level(module) \n");
>         push_log_level(module);
>         printf("before a_crate->module_init_id = module->id \n");
>         a_crate->module_init_id = module->id;
>         printf("before module->props->init_slow(a_crate, module) \n");
>         if (!module->props->init_slow(a_crate, module)) {
>             printf("before pop_log_level(module) \n");
>             pop_log_level(module);
>             printf("before goto crate_init_done \n");
>             goto crate_init_done;
>         }
>         printf("before module_init_id_mark(a_crate, module) \n");
>         module_init_id_mark(a_crate, module);
>         printf("before pop_log_level(module) \n");
>         pop_log_level(module);
>     }
> 
> Thus, to me it looks like the check "if (!module->props->init_slow(a_crate,
> module)) ..." is doing something quite horrible to the RIO4.
> 
> 
> This is unfortunate, because my original aim was to show that there is also
> a bug/mistake in readout_dt of the V560 module. But I did not come this far.
> 
> 
> Do you have any idea what might cause the freezing of the RIO4?
> 
> 
> 
> 
> Best greetings and many thanks
> 
> Günter
> 
> 
> 
> 
> 
>