Skip your coffee break: Speed up MCMC convergence
Question Answer(s)
Is Nimble based on Python? The call to the methods using object$method(arg) looks not like standard R synthax. No Python involved as far as I know.
NA Something like `object$method(arg)` is also R syntax for reference classes, for example.
For CMR models expressed as HMMs, is the model likelihood exactly the same for Stan and nimble? I heard that there are subtle differences with Stan, but I may be wrong. Same I'd say from I understand it from here . Stan also has the marginalized likelihood for CMR models which is available in Nimble through nimbleEcology, which will cover soon. See also Yackulic et al. awesome paper .
NA But you'd better double check with a Stan expert, which I'm definitely not !
NA
NA Counter offer :slightly_smiling_face:
NA And
NA Couldn't find "Kery and Schaub" translated to Python...nobody has done it yet
NA Chris Fonnesbeck might know :wink:
Am I the only one with issues for downloading the R script of Demo 8?
No I have the same problem (Demo 7 ?)
NA should be working fine now, see :
it's called 7 instead of 8? should be working now see :
NA it worked for me before thanks :slightly_smiling_face:
The Demos names are changed, but the link for downloading the Demo 8 (named Demo 7), is not available i think. There is indeed a discrepancy between the class number and the name of the files for live demos. But the links are working I think, see
NA Thanks, this link is working!
I definitely need to run my chains in parallel, and my model is very touchy about inits. What is the best way to ensure different inits for each chain and also that I can save these inits to reproduce the same results? Hi I think you figured this out by now but yes, if your model is "touchy" about inits, it's best to write a tailored function to draw reasonable initial values. I always "pre-sample" initial values (as I have shown in demo 2) and then "test" that they are "good" by running my model with 2 (sometimes 10) iterations, no burnin. If initialization problems occur, you'll get corresponding warnings with 2 iterations as well, but it goes much quicker. Plus, if warnings happen for only 2 (instead of several 1000s of iterations) you can actually read/backtrack them :wink: Hope this helps!
NA I was struggling combining using this with running chains in parallel, but I think I've fixed it now :crossed_fingers: it's still working away, hopefully on the 10 iterations and not the compiling :chains:
NA Great! Also remember to use the setSeed argument for running the MCMC :wink: As a side-note: I don't usually parallelize nimble. If you run chains sequentially, you only need to build, calculate, and compile once, irrespective of how many chains you are going to run. So often for me, the hassle is not worth the time. (But then again, I send most of my models to run on a cluster....)
NA I'm using a cluster too but these ones are spatial capture-recapture models in a really large area with many years of data :exploding_head:
NA Ah I see! I have not done any SCR myself, but check out nimble magic for SCR done by awesome colleagues very recently: May be useful to you :wink:
<@U0203USKJRE> wrote something on parallelization in Nimble <https://r-nimble.org/nimbleExamples/parallelizing_NIMBLE.html> and <@U01UMBR0K38> had a nice example of how to ensure reproducibility when picking initial values at the end of the live demo on Nimble <https://oliviergimenez.github.io/bayesian-cr-workshop/worksheets/2_demo.html>. Hope this helps. yes, I was struggling to put the two together plus custom distributions, samplers... but I think I'm getting somewhere with a very, very long `foreach()`call and a function for inits, hopefully they are saved in the final output. Merci beaucoup
NA Awesome! Feel free to share your workflow :wink:
NA We'll see if it works first :sweat_smile:
I think that should work. The key is that each thread should have its own compiled MCMC and model, and it looks like you've arranged that. A way to check would be to insert some different values somewhere in the models and then check them. E.g. in the first loops you could do `compiled_model[[i]]$a &lt;- i` and in the `foreach` loop you could start with `cat(paste('thread i = ', compiled_model[[i]]$a))` That's just a way to check. I'm not sure what went wrong with other trials you made, but you should be able to do the `nimbleModel`` ...`compileNimble` steps in separate threads too. I am not a parallelization expert, so take it with a grain of salt! Thanks, it was failing to create a shared library, the time to compile not in parallel is probably less than the time that it will take for me to fix it, and as long as I can get chains running in parallel I'll be making big gains. :woman-running: