You need to have the correct color matrix, transfer and primaries data and also the mastering metadata or it won’t work correctly.
The metadata, if any, should be in the Encoding settings part.
For example stuff like “colorprim=9 / transfer=16 / colormatrix=9”, “master-display=G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(40000000,50) / cll=4390,1327”
CLL is not always defined, some releases don’t use the value at all.
You can also see the data like this:
Color range : Limited
Color primaries : BT.2020
Transfer characteristics : PQ
Matrix coefficients : BT.2020 non-constant
Mastering display color primaries : Display P3
Mastering display luminance : min: 0.0050 cd/m2, max: 4000 cd/m2
Maximum Content Light Level : 4390 cd/m2
Maximum Frame-Average Light Level : 1327 cd/m2
The metadata must be set in the x265 commandline, I don’t think it’s possible to add it afterwards.
I think you should compare the data of your encode and the original and see if there is a difference which would explain the other one working and the other one not.