Problems with Images in C#

I am encountering a not-so-happy issue with programming in .NET for something that works with images. Because I write metadata and I don’t just read it, I am encountering issues regarding image quality and compression. If I let the happy little framework do things its way, I end up with files of reduced size. If I change the quality setting to be 100% (which I interpreted as “don’t fuck with it”), I get files that are much larger. I am intending to work only with metadata, not the images themselves, so I am at a loss as to what is going on here. I would really like to be able to directly copy the data from one file to the new file (since .NET requires a new file for each time a modification is done). I am so pissed off right now, since I have been releasing software for the past few weeks that will reduce image quality if allowed to overwrite the originals, and because I wasn’t using real files until now (just my sample images, which are simple geometric forms, under 10KB), I had no clue it was doing it. I knew a long time ago that my first pass (ExIf) was doing it, but I had no clue it was going on with the 2nd pass (XMP).