Bit of a breakthrough!!!
My 2 problem videos were mp4. I muxed them to mkv using mkvmerge and the problem has disappeared. So no need to alter that dynamic_buf_num_margin anymore!!
PS. The universal sample still has the issue (even tried remuxing with mkvmerge) and I can only get it to play smoothly by setting the value to 12 or 16. But my problem movies now work with the default value of 7 (as mkv files). Also, that universal sample plays back very choppily on my Win 10 machine (10th gen i7 laptop).