I have checked several times in different datasets. It looks like the function of MSA for Attribute Agreement does not work accurately
Please email (support@blueskystatistics.com) a sample dataset along with what should be the correct values you are expecting. We will verify the results and see why the discrepancy is.