Results 1 to 21 of 21

Thread: Tips on finding data in save game files?

  1. #1

    Join Date
    29-07-19
    Posts
    41

    Question Tips on finding data in save game files?

    Having a few days off during lockdown I figured I would have a play with my API and my personal tool. I've been trying to figure out two things. And wondered if it would be a good to share how we are approach reverse engineering the code and save files.

    As an example I've documented my process of trying to find where the Nickname you set on a player is stored?

    Step one I had a search on the forum for nickname, which mostly just brought up posts replying to Nick+Co and nothing else that was relevant.

    Next I have a search in some random decompiled java file I have, not sure who it was by (sadimgnik?) and where I got it but gives some good pointers. Anyway it had nothing on nickname, and incorrectly inferred common name as nickname in its usage but it did give me an idea to look into notes.dat as it seems kind of logically these two would be paired. It did have one class on notes which outlined some expected values.

    I took my leap of faith and looked. It was easy enough to figure out the row size, it starts with 128 header with no notes and adding a note increases it to 404 so 128 - 404 = 276 for a single note, which seemed to match up with code above. (I probably should test adding another note increases it by 276). From that 276 I was able to find the offsets for when it was created, the staff id and the note (although offsets didn't match up with code I found on notes...?). Anyway, Happy days.

    I then figured that I could use a compare tool like KDiff3 to compare the data contents between saves to figure out which bits are changing based on the different notes options to find the offsets and hopefully work out the values of what is changing. I created a method which for each index of the array will attempt to convert the index into sbyte, boolean, byte, char, int16, uint16, int32, short date, long date, ascii/latin string and print it out. These are just values I currently process, if I have missed one please say! However doing this the contents was exactly the same between the two saves. Admittedly I probably should of just done the whole byte array to start with... (which is now on my to do list).

    As that was a dead end I figured, hey I could set the nickname to something fairly unique like cuntz (sorry) and then search every .dat/.tmp file, at every index looking for that string in ascii/latin and find the offset like I did with the notes.dat. However doing this it did not find the nickname. I'm fairly confident the search method works as it finds the note I had set earlier easy enough and before I added the 'z' it found Scunthorpe and a random player I had not heard of...

    My next step is to write a method printing out for each .dat/.tmp size to see if any increase adding nicknames but I have a feeling they won't so after that I think I will write a comparison method to compare .dat files between between saves to see if something internally is being set some where.

    It will probably turn out it is set in the binary of the .exe and I've been barking up the wrong tree...

    Would be good to hear if any other users have any thoughts or processes.
    Last edited by tonytony; 12-11-20 at 01:00 PM.

  2. #2

    Join Date
    04-05-20
    Posts
    89
    Quote Originally Posted by tonytony View Post
    Next I have a search in some random decompiled java file I have, not sure who it was by (sadimgnik?) and where I got it but gives some good pointers.
    Wow, that code is really old. I wrote it - Sadimgnik is a reference to: https://www.youtube.com/watch?v=bzB7nqh-O5g

    All my code (https://champman0102.co.uk/downloads.php?do=file&id=201) is released under the GPL, so there's full source code without the need to decompile anything.

    Accessing the header at the top of a save is easy to do (from memory), so finding out which file has changed size should be easy. It's strange that text matching doesn't work, maybe the game uses a different character set for nicknames (where characters are 2 bytes and not 1).

    I might look into this myself later.

  3. #3

    Join Date
    29-07-19
    Posts
    41
    Quote Originally Posted by John Locke View Post
    Wow, that code is really old. I wrote it - Sadimgnik is a reference to: https://www.youtube.com/watch?v=bzB7nqh-O5g
    Note sure that clears it up anymore for me...

    Quote Originally Posted by John Locke View Post
    All my code (https://champman0102.co.uk/downloads.php?do=file&id=201) is released under the GPL, so there's full source code without the need to decompile anything.
    Unfortunately that link, like many others in the forum lead to a dead send space link or the like.

    I took inspiration of your The Java Programming Thread (https://champman0102.co.uk/showthread.php?t=3028&page=2) and started my own .NET API. I do plan to release the source of it at some point.

    Anyway, I looped over the header to give out the details of the tables (.dat/.tmp) and ran it for before and after adding a nickname but didn't see any obvious difference apart from the Position changing, I saved it a few times without doing anything and noticed that the Positions changed each time which isn't ideal, and the Tactics file actually got bigger (possible memory leak?).

    I decided to apply a nickname for the whole team to make it more obvious and low and be hold human_manager.data has gotten bigger (683734 - 684214 = 480). As an educated guess I think we are looking at class with Id, Staff Id, and Name which is 24 characters but that doesn't quite match the memory foot print of the 19 players I set it on (480/19 = 25.263).

    Last edited by tonytony; 12-11-20 at 03:53 PM.

  4. #4

    Join Date
    04-05-20
    Posts
    89
    Try this link: https://champman0102.co.uk/downloads.php?do=file&id=147

    Older version, but might be helpful.

    How long were the nicknames you were using? Could the game be storing staff id, nick name pairs, e.g. if the nickname is 10 characters could it be storing 4 bytes for staff id and 11 characters (10 + null) for the nickname?

  5. #5

    Join Date
    29-07-19
    Posts
    41
    The longest you can enter is 24 characters and I entered variable lengths from min to max. My impression is that the save mechanism isn't smart enough to save things as variable lengths and just blocks out the allocated space regardless if it used or not.

    I think the issue might stem with human_manager.dat, I just checked and when it reads the contents of this the whole byte array is 0, switching to players.dat I have data coming through.

    I wonder if that file is encoded differently.

  6. #6

    Join Date
    04-05-20
    Posts
    89
    The usual storage is certainly a fixed number of bytes, which will help when accessing a large number of entries - but I can see why they might change that for such a small number of likely entries.

    If you output human_manager.dat to a file and upload it I can take a look through it if you want.

  7. #7

    Join Date
    04-05-20
    Posts
    89
    How are you searching the data? I've just tried and found the nickname I set straight away (305160 bytes into human_manager.dat).

  8. The Following User Says Thank You to John Locke For This Useful Post:


  9. #8

    Join Date
    29-07-19
    Posts
    41
    Quote Originally Posted by John Locke View Post
    How are you searching the data? I've just tried and found the nickname I set straight away (305160 bytes into human_manager.dat).
    It was a school boy error on my side My implementation is designed to be reading rows of data, as with human_manager.dat and others I don't know the size of these so it was setting the row to the size of the table, however this number is too big for part of the code and would fail silently. I'm just changing this on my side to fail fast and give an expected result.

    Should add, thank you for looking. Gives me hope that I am in the right direction!
    Last edited by tonytony; 12-11-20 at 06:34 PM.

  10. The Following User Says Thank You to tonytony For This Useful Post:


  11. #9

    Join Date
    04-05-20
    Posts
    89
    The 4 bytes before the nickname text is the staff id of the person.

  12. The Following User Says Thank You to John Locke For This Useful Post:


  13. #10

    Join Date
    04-05-20
    Posts
    89
    The 4 bytes before that, in my example, contains the value: 1 - which suggests it is the number of entries / people with nicknames.

  14. The Following User Says Thank You to John Locke For This Useful Post:


  15. #11

    Join Date
    29-07-19
    Posts
    41
    Quote Originally Posted by John Locke View Post
    The 4 bytes before the nickname text is the staff id of the person.
    I'm seeing the same.

    The bigger issue now is that I am suspecting that there is multiple class types in this file. Given the size, and generic name. I had looked into the index.dat but don't remember seeing any entry for this file. I wonder if it holds its own header information.

  16. #12

    Join Date
    04-05-20
    Posts
    89
    index.dat only stores ~20 files (those in the Data directory), everything else is just the main header (and yes, they may have an internal header).

    The first 4 bytes of the file are 16 0 0 0 (decimal) - which (coincidentally?) is the max number of managers.

  17. The Following User Says Thank You to John Locke For This Useful Post:


  18. #13

    Join Date
    18-07-15
    Posts
    795
    @tonytony: Can you not use code from my Patcher to assist with some of this: https://github.com/nckstwrt/CM0102Pa...SaveReader2.cs and https://github.com/nckstwrt/CM0102Pa.../Structures.cs ?

  19. The Following User Says Thank You to Nick+Co For This Useful Post:


  20. #14

    Join Date
    17-06-12
    Posts
    455
    i know many things from .sav

    i know how to edit future transfers, edit already made transfers (e.g Lazio buying Almeyda for 66 000 000 euros which is nonsense), edit any results or winners of world player of the year etc.

    too bad i dont know javascript and couldnt write a programm, so all that stuff i am doing straight in file.

  21. #15

    Join Date
    29-07-19
    Posts
    41
    Quote Originally Posted by Nick+Co View Post
    @tonytony: Can you not use code from my Patcher to assist with some of this: https://github.com/nckstwrt/CM0102Pa...SaveReader2.cs and https://github.com/nckstwrt/CM0102Pa.../Structures.cs ?
    Definitely useful, my implementation is closer to your original SaveReader.cs, which I lifted from some Delphi code (I think), however your SaveReader2.cs looks cleaner implementation. Unfortunately your structures don't cover the file I'm looking at.

    Quote Originally Posted by John Locke View Post
    The 4 bytes before that, in my example, contains the value: 1 - which suggests it is the number of entries / people with nicknames.
    You are correct what I am seeing is;

    Initial int for the number of nicknames that manager has set, and then for each entry an int for Staff Id and then 26 bytes for the nickname. I want to say 25 then a null indicator?

    Quote Originally Posted by John Locke View Post
    The first 4 bytes of the file are 16 0 0 0 (decimal) - which (coincidentally?) is the max number of managers.
    It would of taken me a while to notice that, if it is true but gave me some ideas. Did some further investigation and adding more managers didn't alter the size of this particular file.

    Saving nicknames of players on different managers saved them in different parts of the file but for the same manager in the same area so I suspect there is 16 class objects for managers in this file.

    Next I will add a single nickname for 16 players for 16 managers. From this I can see the spacing between the entries of nicknames to get a rough size of the objects to objects. Hopefully this is all fixed sized. I will then add some more nicknames per manager and confirm sizes to confirm this.

  22. The Following User Says Thank You to tonytony For This Useful Post:


  23. #16

    Join Date
    18-07-15
    Posts
    795
    @tonytony: Sorry Tony, I didn't read this thread properly - you are mainly concerned about Nick Names. John Locke is right - get the human_manager.dat, seek to 305152 (4A800). Read a DWORD (4 bytes) to see how many nicknames there are. Then it's a DWORD for the staff id and then 26 bytes for the name. So pseudo code would be:
    Code:
    file = GetFile("human_manager.dat")
    file.SeekInFile(0x4A800)
    NumberOfNicknames = file.Read4ByteInteger()
    for ( i = 0; i < NumberOfNicknames; i++)
    {
         staffId = file.Read4ByteInteger()
         nameName = file.Read26ByteString() 
    }

  24. The Following 2 Users Say Thank You to Nick+Co For This Useful Post:


  25. #17

    Join Date
    29-07-19
    Posts
    41
    Quote Originally Posted by Nick+Co View Post
    @tonytony: Sorry Tony, I didn't read this thread properly - you are mainly concerned about Nick Names. John Locke is right - get the human_manager.dat, seek to 305152 (4A800). Read a DWORD (4 bytes) to see how many nicknames there are. Then it's a DWORD for the staff id and then 26 bytes for the name.
    Tbf the title is a tad miss leading now as I've gone a bit of piste. I guess the aim was more to find out ways other people try to figure out missing file locations that are generally unknown.

    So for example, what is your thought process on it being DWORD, (I guess in my context UInt32).

    As John Locke mentioned the count was -1, DWORD would not be the variable type because of its accepted range?
    Last edited by tonytony; 13-11-20 at 04:37 PM.

  26. #18

    Join Date
    18-07-15
    Posts
    795
    @tonytony: It's clearly 4 bytes - whether it's signed or not makes little difference - 0XFFFFFFFF is as familiar as -1 to many folk Where did John Lock say it's -1? If there's no nick names in the game it will be 0 in that location - not -1/ffffffff.
    (Many Windows APIs return DWORDs - but an error will be listed as returning -1 (0XFFFFFFFF) - GetFileAttributes is an example off the top of my head. But we're digressing )

  27. #19

    Join Date
    29-07-19
    Posts
    41
    @Nick+Co That was an error on my part, I miss read "1 -" by Lock early in the thread for -1 which would make me think it wasn't a DWORD as its range is from 0 and can't be minus but I you are probably right as it doesn't make sense to ever be minus in this situation. Digressing is good. Sharing knowledge etc.

    I added 16 managers and then a nickname for a player on each and found that the space between each occurrence is 1902.

    Interestingly adding managers changed the previous index of 305152.

  28. #20

    Join Date
    18-07-15
    Posts
    795
    @tonytony: Urgh - you're right! this is more tricky! here's some code that does seem to parse the nicknames out, but the process and code is awful:
    Code:
                Encoding latin1 = Encoding.GetEncoding("ISO-8859-1");
                using (var fs = File.Open(@"c:\downloads\human_manager6.dat", FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
                using (var br = new BinaryReader(fs))
                {
                    bool firstHeaderInt = true;
                    while (true)
                    {
                        if (fs.Position >= fs.Length)
                            break;
                        var headerInt = br.ReadInt32();
                        if (headerInt == -1 || headerInt == 0 || firstHeaderInt)
                        {
                            var nextInt = br.ReadInt32();
                            if (nextInt == 0)
                                fs.Seek(0xCE6 - 8, SeekOrigin.Current);
                            else
                                fs.Seek(0x1eCD - 8, SeekOrigin.Current);
                        }
                        else
                        {
                            // Must be a nickName
                            for (int i = 0; i < headerInt; i++)
                            {
                                var staffId = br.ReadInt32();
                                var textBytes = br.ReadBytes(26);
                                var nickName = latin1.GetString(textBytes, 0, Array.IndexOf(textBytes, (byte)0));
                                Console.WriteLine("{0} = {1}", staffId, nickName);
                            }
                            fs.Seek(0x1eCD - 4, SeekOrigin.Current);
                        }
    
    
                        firstHeaderInt = false;
                    }
                }

  29. The Following User Says Thank You to Nick+Co For This Useful Post:


  30. #21

    Join Date
    29-07-19
    Posts
    41
    I think that is the best attempt for now. Worth noting it seems that the 1st manager is at the end of the file.

    I had hoped to find some other information stored within human_manager.dat, or something referenced in other files but I seem to have hit a dead end with any ideas.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •