<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta name="Generator" content="Microsoft Exchange Server">
<!-- converted from text --><style><!-- .EmailQuote { margin-left: 1pt; padding-left: 4pt; border-left: #800000 2px solid; } --></style>
</head>
<body>
<meta content="text/html; charset=UTF-8">
<style type="text/css" style="">
<!--
p
{margin-top:0;
margin-bottom:0}
-->
</style>
<div dir="ltr">
<div id="x_divtagdefaultwrapper" dir="ltr" style="font-size:14pt; color:#000000; font-family:Calibri,Helvetica,sans-serif">
<p>Dear <span>Hċkan</span>,</p>
<p><br>
</p>
<p>thanks for the hint to flush and sleep. Indeed, I now see that the crash happens in init_slow of V560 at this line:</p>
<p><br>
</p>
<p></p>
<div style="color:#cccccc; background-color:#1f1f1f; font-family:Consolas,'Courier New',monospace; font-weight:normal; font-size:14px; line-height:19px; white-space:pre">
<div><span style="color:#cccccc"> </span><span style="color:#9cdcfe">v560</span><span style="color:#cccccc">-></span><span style="color:#9cdcfe">sicy_map</span><span style="color:#cccccc">
</span><span style="color:#d4d4d4">=</span><span style="color:#cccccc"> </span><span style="color:#dcdcaa">map_map</span><span style="color:#cccccc">(</span><span style="color:#9cdcfe">v560</span><span style="color:#cccccc">-></span><span style="color:#9cdcfe">address</span><span style="color:#cccccc">,
</span><span style="color:#569cd6">MAP_SIZE</span><span style="color:#cccccc">, </span>
<span style="color:#4fc1ff">KW_NOBLT</span><span style="color:#cccccc">,</span></div>
<div><span style="color:#cccccc"> </span><span style="color:#b5cea8">0</span><span style="color:#cccccc">,
</span><span style="color:#b5cea8">0</span><span style="color:#cccccc">, </span><span style="color:#569cd6">MAP_POKE_ARGS</span><span style="color:#cccccc">(fixed_code),
</span><span style="color:#569cd6">MAP_POKE_ARGS</span><span style="color:#cccccc">(scale_clear));</span></div>
</div>
<p></p>
<p><br>
</p>
<p>Maybe the code is accessing/writing into a memory location that it should better not touch?<br>
</p>
<p>This problematic line is then followed by:</p>
<p><br>
</p>
<p></p>
<div style="color:#cccccc; background-color:#1f1f1f; font-family:Consolas,'Courier New',monospace; font-weight:normal; font-size:14px; line-height:19px; white-space:pre">
<div><span style="color:#cccccc"> </span><span style="color:#9cdcfe">id</span><span style="color:#cccccc">
</span><span style="color:#d4d4d4">=</span><span style="color:#cccccc"> </span><span style="color:#569cd6">MAP_READ</span><span style="color:#cccccc">(</span><span style="color:#9cdcfe">v560</span><span style="color:#cccccc">-></span><span style="color:#9cdcfe">sicy_map</span><span style="color:#cccccc">,
fixed_code);</span></div>
</div>
<br>
<p></p>
<p>The corresponding line in the V560 code on the system that was running with this module looks like this:</p>
<p><br>
</p>
<p></p>
<div style="color:#cccccc; background-color:#1f1f1f; font-family:Consolas,'Courier New',monospace; font-weight:normal; font-size:14px; line-height:19px; white-space:pre">
<div><span style="color:#cccccc"> </span><span style="color:#9cdcfe">v560</span><span style="color:#cccccc">-></span><span style="color:#9cdcfe">sicy_map</span><span style="color:#cccccc">
</span><span style="color:#d4d4d4">=</span><span style="color:#cccccc"> </span><span style="color:#dcdcaa">map_map</span><span style="color:#cccccc">(</span><span style="color:#9cdcfe">v560</span><span style="color:#cccccc">-></span><span style="color:#9cdcfe">address</span><span style="color:#cccccc">,
</span><span style="color:#dcdcaa">MAP_SIZE_MAX</span><span style="color:#cccccc">(</span><span style="color:#d4d4d4">*</span><span style="color:#cccccc">v560), KW_NOBLT,</span></div>
<div><span style="color:#cccccc"> </span><span style="color:#b5cea8">0</span><span style="color:#cccccc">,
</span><span style="color:#b5cea8">0</span><span style="color:#cccccc">, </span><span style="color:#dcdcaa">MAP_POKE_ARGS</span><span style="color:#cccccc">(</span><span style="color:#d4d4d4">*</span><span style="color:#9cdcfe">v560</span><span style="color:#cccccc">-></span><span style="color:#9cdcfe">read</span><span style="color:#cccccc">,
fixed_code),</span></div>
<div><span style="color:#cccccc"> </span><span style="color:#dcdcaa">MAP_POKE_ARGS</span><span style="color:#cccccc">(</span><span style="color:#d4d4d4">*</span><span style="color:#9cdcfe">v560</span><span style="color:#cccccc">-></span><span style="color:#9cdcfe">write</span><span style="color:#cccccc">,
scale_clear));</span></div>
</div>
<br>
<p></p>
<p>And is followed by:</p>
<p><br>
</p>
<p></p>
<div style="color:#cccccc; background-color:#1f1f1f; font-family:Consolas,'Courier New',monospace; font-weight:normal; font-size:14px; line-height:19px; white-space:pre">
<div><span style="color:#cccccc"> mapped_ptr </span><span style="color:#d4d4d4">=</span><span style="color:#cccccc">
</span><span style="color:#dcdcaa">map_get_mapped_ptr</span><span style="color:#cccccc">(</span><span style="color:#9cdcfe">v560</span><span style="color:#cccccc">-></span><span style="color:#9cdcfe">sicy_map</span><span style="color:#cccccc">);</span></div>
<div><span style="color:#cccccc"> </span><span style="color:#9cdcfe">v560</span><span style="color:#cccccc">-></span><span style="color:#9cdcfe">read</span><span style="color:#cccccc">
</span><span style="color:#d4d4d4">=</span><span style="color:#cccccc"> mapped_ptr;</span></div>
<div><span style="color:#cccccc"> </span><span style="color:#9cdcfe">v560</span><span style="color:#cccccc">-></span><span style="color:#9cdcfe">write</span><span style="color:#cccccc">
</span><span style="color:#d4d4d4">=</span><span style="color:#cccccc"> mapped_ptr;</span></div>
</div>
<br>
<p></p>
<p>Maybe you already have an idea what causes the problem here?</p>
<p><br>
</p>
<p>I will now go to the system that was running with V560 and make a push of the NURDLIB.</p>
<p><br>
</p>
<p><br>
</p>
<p><br>
</p>
<p>Best greetings</p>
<p>Günter<br>
</p>
<p><br>
</p>
<br>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>Von:</b> subexp-daq <subexp-daq-bounces@lists.chalmers.se> im Auftrag von Hċkan T Johansson <f96hajo@chalmers.se><br>
<b>Gesendet:</b> Dienstag, 20. Februar 2024 20:13:32<br>
<b>An:</b> Discuss use of Nurdlib, TRLO II, drasi and UCESB.<br>
<b>Betreff:</b> Re: [subexp-daq] Report of a possible bug of the CAEN_V560 module</font>
<div> </div>
</div>
</div>
<font size="2"><span style="font-size:10pt;">
<div class="PlainText"><br>
Dear Günter,<br>
<br>
I took the files you provided and for comparison put them in a branch <br>
'old_caen_v560'.<br>
<br>
git diff origin/old_caen_v560..origin/master<br>
<br>
however does not show anything which is suspicious to me. Perhaps Hans <br>
can spot something.<br>
<br>
Otherwise, the only idea I can come up with is to continue to bisect the <br>
code inside slow init.<br>
<br>
However, before that, I would suggest to add<br>
<br>
fflush(stdout); sleep(1);<br>
<br>
after each printf statement, such that one can be quite sure that the <br>
printout is not eaten when the RIO crash happens. I.e. that it actually <br>
had gotten further than shown by the prints.<br>
<br>
Best regards,<br>
Hċkan<br>
<br>
<br>
<br>
<br>
On Tue, 20 Feb 2024, Weber, Guenter Dr. wrote:<br>
<br>
> <br>
> Dear friends,<br>
> <br>
> <br>
> I now had a look at the system where the V560 was running. It was also setup<br>
> by Bastian. And there the code for the V560 module is slightly different<br>
> from the one included in the NURDLIB branch that I am using on the test<br>
> system.<br>
> <br>
> <br>
> Maybe you can have a look at it.<br>
> <br>
> <br>
> I also could push the complete NURDLIB from this system, if this helps.<br>
> <br>
> <br>
> <br>
> <br>
> Best greetings<br>
> <br>
> Günter<br>
> <br>
> <br>
> <br>
> <br>
> ____________________________________________________________________________<br>
> Von: subexp-daq <subexp-daq-bounces@lists.chalmers.se> im Auftrag von Weber,<br>
> Guenter Dr. <g.weber@hi-jena.gsi.de><br>
> Gesendet: Dienstag, 20. Februar 2024 10:58:27<br>
> An: Discuss use of Nurdlib, TRLO II, drasi and UCESB.<br>
> Betreff: [subexp-daq] Report of a possible bug of the CAEN_V560 module <br>
> <br>
> Dear friends,<br>
> <br>
> <br>
> I now grabbed a V560 module that was working fine in another DAQ system and<br>
> put it into our test system.<br>
> <br>
> <br>
> The main.cfg looks like this:<br>
> <br>
> <br>
> log_level=spam # info, verbose, debug, spam<br>
> <br>
> CRATE("MCAL") {<br>
> GSI_VULOM(0x03000000) {<br>
> timestamp = true # needed to get timestamps in the data output<br>
> # ecl=0..15<br>
> }<br>
> BARRIER<br>
> CAEN_V560(0x333333300) {<br>
> use_veto = true<br>
> } <br>
> # CAEN_V767A(0x03100000) {<br>
> # }<br>
> }<br>
> <br>
> Starting the DAQ now results in a freeze of the RIO4. A reset of the crate<br>
> is necessary to talk to it again.<br>
> <br>
> <br>
> The problem occurs in the first slow init of the V560 module. To find the<br>
> exact line, I added some output to CRATE.C:<br>
> <br>
> <br>
> 10: crate/crate.c:923: .Slow-init module[0]=GSI_VULOM.<br>
> before push_log_level(module)<br>
> before a_crate->module_init_id = module->id<br>
> before module->props->init_slow(a_crate, module)<br>
> LOG: TRLO: MD5SUM: 0x1409285e (CT: 63bb1d44 = 2023-01-08 19:45:08 UTC)<br>
> before module_init_id_mark(a_crate, module)<br>
> before pop_log_level(module)<br>
> 10: crate/crate.c:923: .Slow-init module[1]=CAEN_V560.<br>
> before push_log_level(module)<br>
> before a_crate->module_init_id = module->id<br>
> before module->props->init_slow(a_crate, module)<br>
> <br>
> <br>
> The CRATE.C code now looks like this:<br>
> <br>
> <br>
> TAILQ_FOREACH(module, &a_crate->module_list, next) {<br>
> if (NULL == module->props) {<br>
> continue;<br>
> }<br>
> LOGF(info)(LOGL, "Slow-init module[%u]=%s.", module->id,<br>
> keyword_get_string(module->type));<br>
> printf("before push_log_level(module) \n");<br>
> push_log_level(module);<br>
> printf("before a_crate->module_init_id = module->id \n");<br>
> a_crate->module_init_id = module->id;<br>
> printf("before module->props->init_slow(a_crate, module) \n");<br>
> if (!module->props->init_slow(a_crate, module)) {<br>
> printf("before pop_log_level(module) \n");<br>
> pop_log_level(module);<br>
> printf("before goto crate_init_done \n");<br>
> goto crate_init_done;<br>
> }<br>
> printf("before module_init_id_mark(a_crate, module) \n");<br>
> module_init_id_mark(a_crate, module);<br>
> printf("before pop_log_level(module) \n");<br>
> pop_log_level(module);<br>
> }<br>
> <br>
> Thus, to me it looks like the check "if (!module->props->init_slow(a_crate,<br>
> module)) ..." is doing something quite horrible to the RIO4.<br>
> <br>
> <br>
> This is unfortunate, because my original aim was to show that there is also<br>
> a bug/mistake in readout_dt of the V560 module. But I did not come this far.<br>
> <br>
> <br>
> Do you have any idea what might cause the freezing of the RIO4?<br>
> <br>
> <br>
> <br>
> <br>
> Best greetings and many thanks<br>
> <br>
> Günter<br>
> <br>
> <br>
> <br>
> <br>
> <br>
></div>
</span></font>
</body>
</html>