[subexp-daq] Report of a possible bug of the CAEN_V560 module

[Weber, Guenter Dr.]

Dear Håkan,


thanks for the hint to flush and sleep. Indeed, I now see that the crash happens in init_slow of V560 at this line:


    v560->sicy_map = map_map(v560->address, MAP_SIZE, KW_NOBLT,
        0, 0, MAP_POKE_ARGS(fixed_code), MAP_POKE_ARGS(scale_clear));


Maybe the code is accessing/writing into a memory location that it should better not touch?

This problematic line is then followed by:


    id = MAP_READ(v560->sicy_map, fixed_code);


The corresponding line in the V560 code on the system that was running with this module looks like this:


    v560->sicy_map = map_map(v560->address, MAP_SIZE_MAX(*v560), KW_NOBLT,
        0, 0, MAP_POKE_ARGS(*v560->read, fixed_code),
        MAP_POKE_ARGS(*v560->write, scale_clear));


And is followed by:


    mapped_ptr = map_get_mapped_ptr(v560->sicy_map);
    v560->read = mapped_ptr;
    v560->write = mapped_ptr;


Maybe you already have an idea what causes the problem here?


I will now go to the system that was running with V560 and make a push of the NURDLIB.




Best greetings

Günter


________________________________
Von: subexp-daq <subexp-daq-bounces at lists.chalmers.se> im Auftrag von Håkan T Johansson <f96hajo at chalmers.se>
Gesendet: Dienstag, 20. Februar 2024 20:13:32
An: Discuss use of Nurdlib, TRLO II, drasi and UCESB.
Betreff: Re: [subexp-daq] Report of a possible bug of the CAEN_V560 module


Dear Günter,

I took the files you provided and for comparison put them in a branch
'old_caen_v560'.

git diff origin/old_caen_v560..origin/master

however does not show anything which is suspicious to me.  Perhaps Hans
can spot something.

Otherwise, the only idea I can come up with is to continue to bisect the
code inside slow init.

However, before that, I would suggest to add

  fflush(stdout); sleep(1);

after each printf statement, such that one can be quite sure that the
printout is not eaten when the RIO crash happens.  I.e. that it actually
had gotten further than shown by the prints.

Best regards,
Håkan




On Tue, 20 Feb 2024, Weber, Guenter Dr. wrote:

>
> Dear friends,
>
>
> I now had a look at the system where the V560 was running. It was also setup
> by Bastian. And there the code for the V560 module is slightly different
> from the one included in the NURDLIB branch that I am using on the test
> system.
>
>
> Maybe you can have a look at it.
>
>
> I also could push the complete NURDLIB from this system, if this helps.
>
>
>
>
> Best greetings
>
> Günter
>
>
>
>
> ____________________________________________________________________________
> Von: subexp-daq <subexp-daq-bounces at lists.chalmers.se> im Auftrag von Weber,
> Guenter Dr. <g.weber at hi-jena.gsi.de>
> Gesendet: Dienstag, 20. Februar 2024 10:58:27
> An: Discuss use of Nurdlib, TRLO II, drasi and UCESB.
> Betreff: [subexp-daq] Report of a possible bug of the CAEN_V560 module
>
> Dear friends,
>
>
> I now grabbed a V560 module that was working fine in another DAQ system and
> put it into our test system.
>
>
> The main.cfg looks like this:
>
>
> log_level=spam # info, verbose, debug, spam
>
> CRATE("MCAL") {
>     GSI_VULOM(0x03000000) {
>         timestamp = true # needed to get timestamps in the data output
>     #   ecl=0..15
>     }
>     BARRIER
>     CAEN_V560(0x333333300) {
>         use_veto = true
>     }
> #   CAEN_V767A(0x03100000) {
> #   }
> }
>
> Starting the DAQ now results in a freeze of the RIO4. A reset of the crate
> is necessary to talk to it again.
>
>
> The problem occurs in the first slow init of the V560 module. To find the
> exact line, I added some output to CRATE.C:
>
>
> 10: crate/crate.c:923: .Slow-init module[0]=GSI_VULOM.
> before push_log_level(module)
> before a_crate->module_init_id = module->id
> before module->props->init_slow(a_crate, module)
> LOG: TRLO: MD5SUM: 0x1409285e (CT: 63bb1d44 = 2023-01-08 19:45:08 UTC)
> before module_init_id_mark(a_crate, module)
> before pop_log_level(module)
> 10: crate/crate.c:923: .Slow-init module[1]=CAEN_V560.
> before push_log_level(module)
> before a_crate->module_init_id = module->id
> before module->props->init_slow(a_crate, module)
>
>
> The CRATE.C code now looks like this:
>
>
>     TAILQ_FOREACH(module, &a_crate->module_list, next) {
>         if (NULL == module->props) {
>             continue;
>         }
>         LOGF(info)(LOGL, "Slow-init module[%u]=%s.", module->id,
>             keyword_get_string(module->type));
>         printf("before push_log_level(module) \n");
>         push_log_level(module);
>         printf("before a_crate->module_init_id = module->id \n");
>         a_crate->module_init_id = module->id;
>         printf("before module->props->init_slow(a_crate, module) \n");
>         if (!module->props->init_slow(a_crate, module)) {
>             printf("before pop_log_level(module) \n");
>             pop_log_level(module);
>             printf("before goto crate_init_done \n");
>             goto crate_init_done;
>         }
>         printf("before module_init_id_mark(a_crate, module) \n");
>         module_init_id_mark(a_crate, module);
>         printf("before pop_log_level(module) \n");
>         pop_log_level(module);
>     }
>
> Thus, to me it looks like the check "if (!module->props->init_slow(a_crate,
> module)) ..." is doing something quite horrible to the RIO4.
>
>
> This is unfortunate, because my original aim was to show that there is also
> a bug/mistake in readout_dt of the V560 module. But I did not come this far.
>
>
> Do you have any idea what might cause the freezing of the RIO4?
>
>
>
>
> Best greetings and many thanks
>
> Günter
>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.chalmers.se/pipermail/subexp-daq/attachments/20240221/994bfd7c/attachment-0001.html>