Page 1 of 11 12345 ... LastLast
Results 1 to 25 of 267

Thread: CM01/02 Updater (a DB/Pre-Game Editor)

  1. #1
    Join Date
    14-06-14
    Location
    the English Channel
    Posts
    212
    vCash
    0

    CM01/02 Updater (a DB/Pre-Game Editor)

    Over the past two weeks I have been hard at work porting over my Eastside Hockey Manager 2007 editing tool (aka EHM 2007 Updater) to CM01/02. Many thanks to Hugo9 who posted the CM database structure here, enabling me to start porting over my tool. If you want to see the sort of things my EHM 2007 Updater can do (so you can see what sort of things I hope to port over to the CM01/02 Updater) then there's a changelist and some screenshots here (the screenshots are slightly out of date now): http://www.ehmtheblueline.com/forums...157635#p157635

    I'm hoping to release the first version of the Updater soon, but thought I'd post a couple of screenshots and a brief update seeing as progress is going really well. Here's what I've done thus far:

    - Loading and saving the database
    - Exporting various data. You can export either to CSV or XML (you can choose from the Settings menu). CSV has smaller file sizes, but XML creates tidier spreadsheets (basically the same sort of formatting as XLS/XLSX spreadsheets).
    - Adjusting all years in the DB (I just need to extend it to the dates stored in the cfg files).


    Things I'll be doing next:

    - Ability to make players and/or non-players free agents.
    - Ability to delete various parts of the database (e.g. delete all staff or staff career history) - including the ability to generate a blank database (but it won't delete nations, cities, teams, etc as these are essential for the game to work).
    - Ability to edit existing data via CSV/XML spreadsheets (e.g. create a list of attributes you want to edit from existing players in a spreadsheet and the Updater will apply them to the DB).
    - Ability to import new data from CSV/XML spreadsheets (e.g. create new staff, import staff career history, etc).

    Here are a couple of early screenshots:





    The tool is compatible with Windows XP onwards.

    Finally, just to say a MASSIVE THANK YOU to the likes of Taipani, John Locke and the guys that run the site and the DB updates. Thanks to you, I have re-discovered the joys of CM01/02 so many years after I last played it.
    Last edited by archibalduk; 05-07-14 at 09:07 PM.


  2. #2
    Join Date
    11-09-12
    Posts
    584
    vCash
    1000
    Quote Originally Posted by archibalduk View Post
    - Ability to import new data from CSV/XML spreadsheets (e.g. create new staff, import staff career history, etc).
    This will be fantastic Great work

  3. The Following User Says Thank You to Zé For This Useful Post:


  4. #3
    Join Date
    18-12-11
    Location
    On the oche
    Posts
    2,897
    vCash
    10736
    Good stuff Archie, i'm sure this will be a very helpful tool for our update team Oh, and welcome to the site

  5. #4
    Join Date
    03-03-12
    Posts
    2,577
    vCash
    5200
    Wow, has there ever been a more exciting time on the forum, Tapani, John Locke & Archibalduk all working their magic.

    From a personal point of view, if this comes off and you can extract/inject to a CSV or XML document, the possibility of a fully updated 0001 is nearer to a reality than ever before.

    Great work Archie.





  6. The Following 2 Users Say Thank You to Cam F For This Useful Post:


  7. #5
    Jesus is offline Official Challenge Team
    Social Media Team
    Join Date
    04-03-12
    Posts
    3,534
    vCash
    681
    Top, top stuff Archie. You're an asset to the forum already with this magic! I don't think words can explain how key this sort of thing can be for the game and the future.

  8. The Following 5 Users Say Thank You to Jesus For This Useful Post:


  9. #6
    Join Date
    04-12-13
    Location
    Latvija
    Posts
    75
    vCash
    500
    It looks promising, but I just truly hope that to use this program I won't need to have admin rights...

  10. #7
    Join Date
    21-06-14
    Location
    Canada
    Posts
    489
    vCash
    500
    Quote Originally Posted by Edgars View Post
    It looks promising, but I just truly hope that to use this program I won't need to have admin rights...
    One way to possibly avoid that is to install CM somewhere that is NOT on C: drive.

    Having used the EHM version of this tool extensively I can say with confidence that this will be great.

  11. #8
    Join Date
    21-06-14
    Location
    Canada
    Posts
    489
    vCash
    500
    Archie - a question/suggestion: are you planning to include that 'count' screen that shows the number of players in each league with minimum, maximum and average CAs? I think that could also be quite useful here too.

  12. #9
    Join Date
    04-12-13
    Location
    Latvija
    Posts
    75
    vCash
    500
    Quote Originally Posted by Rapinoe View Post
    One way to possibly avoid that is to install CM somewhere that is NOT on C: drive.

    Having used the EHM version of this tool extensively I can say with confidence that this will be great.
    I am running the game and original editor without admin rights and without any issues on Windows 7, so that won't be a problem!

  13. #10
    Join Date
    02-03-12
    Posts
    4,818
    vCash
    1075
    archibald, since this tool can be very very helpful for the update team, can you also add any features requested by the team?

    if so, you and the team could cooperate to make this tool as best we can

  14. #11
    Join Date
    14-06-14
    Location
    the English Channel
    Posts
    212
    vCash
    0
    Quote Originally Posted by Edgars View Post
    It looks promising, but I just truly hope that to use this program I won't need to have admin rights...
    If your database is located in C:/Program Files(x86)/.../ then yes you'd need to use Run as Admin (otherwise you end up with ghost copies of your DB in your Roaming folder).

    BUT, you can easily avoid this by coping your DB (i.e. all of your .dat files plus player_setup.cfg) to another folder temporarily. E.g. if you move it to a folder on your Desktop or My Documents then you don't need to use Run as Admin. It's only when it's in Program Files where you have to worry about admin rights.

    Quote Originally Posted by Rapinoe View Post
    Archie - a question/suggestion: are you planning to include that 'count' screen that shows the number of players in each league with minimum, maximum and average CAs? I think that could also be quite useful here too.
    Will do! I'll get the Free Agent Utility and Delete DB functions done and then I'll move on to that for you.

    Quote Originally Posted by milo View Post
    archibald, since this tool can be very very helpful for the update team, can you also add any features requested by the team?

    if so, you and the team could cooperate to make this tool as best we can
    Yes, I would be more than happy to do so. The aim of my tool is to make DB editing easier for the guys who maintain the excellent roster updates. So please do post suggestions and I'll see what I can do. I won't make any promises, but it would be really useful to know what sorts of things would help the roster update team because then I can prioritise those aspects.

    I must however stress that I do have a heavy workload and so progress on this tool will be very stop-start. Just to give you an idea of the other things I have on my plate: 1) I'm continuing to develop my EHM Updater, 2) I help maintain an unofficial roster update for EHM 2007, 3) I'm a researcher for Franchise Hockey Manager and 4) I run my own EHM/FHM website. On top of that, I have a pretty busy job. So I tend to rotate my tasks somewhat. My current priority is to get some basic functions up and running for the CM Updater (at which point I'll release it for testing) and then I'll probably do some roster work and EHM Updater work before returning to spend some more time on the CM Updater. I'm also working on creating a shared library of code that both the CM and the EHM Updater will share - so when I improve core functions for one Updater, the other Updater will benefit too.

  15. The Following 5 Users Say Thank You to archibalduk For This Useful Post:


  16. #12
    Join Date
    04-12-13
    Location
    Latvija
    Posts
    75
    vCash
    500
    Quote Originally Posted by archibalduk View Post
    If your database is located in C:/Program Files(x86)/.../ then yes you'd need to use Run as Admin (otherwise you end up with ghost copies of your DB in your Roaming folder).

    BUT, you can easily avoid this by coping your DB (i.e. all of your .dat files plus player_setup.cfg) to another folder temporarily. E.g. if you move it to a folder on your Desktop or My Documents then you don't need to use Run as Admin. It's only when it's in Program Files where you have to worry about admin rights.
    Glad to read it and thank you for your job!

  17. #13
    Join Date
    30-06-14
    Posts
    420
    vCash
    1500
    Seems promising :-)

    Is one of the intended usages to allow the update team to work in parallell? I.e. make it possible to, say, let one updater take care of Italian leagues and another one Spain -- and sometime before release, dump those leagues as CSV files, and then import them into a master database?

  18. The Following 3 Users Say Thank You to Tapani For This Useful Post:


  19. #14
    Join Date
    14-06-14
    Location
    the English Channel
    Posts
    212
    vCash
    0
    Quote Originally Posted by Tapani View Post
    Seems promising :-)

    Is one of the intended usages to allow the update team to work in parallell? I.e. make it possible to, say, let one updater take care of Italian leagues and another one Spain -- and sometime before release, dump those leagues as CSV files, and then import them into a master database?
    Yes, that's exactly right.

    Each researcher could work on their own blank spreadsheets. They just list the attribute/transfer changes and players/non-players that need to be created in their own spreadsheets and send them to the person who has the DB. The person who has the DB imports the spreadsheets into the DB using the Updater. This is how we have been editing EHM for about 3 years now - we have been able to achieve a lot more than just one person having to make all the changes manually using the Pre-Game Editor.

    Here are some screenshots of some of the EHM Updater spreadsheets we use (they might give you an idea of how the spreadsheets work in the EHM Updater and roughly how I plan to get them to work in the CM Updater):




    In the screenshot above, when this spreadsheet is imported into a DB using the Updater: Andreas Nodl's date of birth will be changed to 1st April and the Stickhandling and Wristshot attributes for Thomas Pock will be changed to 14 and 20 respectively.




    In the screenshot above, when this spreadsheet is imported into a DB using the Updater:

    * Matt Walker will be moved to the New York Rangers' roster and his salary will be set to $5m per year;
    * Brent Burns will be moved to the Worcester Sharks' roster;
    * Filip Kuba will be retired (and set to a free agent);
    * Todd McLellan will be set to a free agent (UFA = free agent in ice hockey terminology).




    The screenshot above is of the staff_new.xml/csv spreadsheet which is used for inserting new players/non-players into the DB.

  20. The Following 6 Users Say Thank You to archibalduk For This Useful Post:


  21. #15
    Join Date
    02-03-12
    Posts
    4,818
    vCash
    1075
    Quote Originally Posted by archibalduk View Post
    Yes, I would be more than happy to do so. The aim of my tool is to make DB editing easier for the guys who maintain the excellent roster updates. So please do post suggestions and I'll see what I can do. I won't make any promises, but it would be really useful to know what sorts of things would help the roster update team because then I can prioritise those aspects.

    I must however stress that I do have a heavy workload and so progress on this tool will be very stop-start. Just to give you an idea of the other things I have on my plate: 1) I'm continuing to develop my EHM Updater, 2) I help maintain an unofficial roster update for EHM 2007, 3) I'm a researcher for Franchise Hockey Manager and 4) I run my own EHM/FHM website. On top of that, I have a pretty busy job. So I tend to rotate my tasks somewhat. My current priority is to get some basic functions up and running for the CM Updater (at which point I'll release it for testing) and then I'll probably do some roster work and EHM Updater work before returning to spend some more time on the CM Updater. I'm also working on creating a shared library of code that both the CM and the EHM Updater will share - so when I improve core functions for one Updater, the other Updater will benefit too.
    we all work here in our spare time

    thank you very much, hope we can really collaborate

  22. #16
    Join Date
    18-12-11
    Location
    Dresden
    Posts
    7,673
    vCash
    4030
    Great stuff mate. I am looking forward to testing it.

  23. #17
    Join Date
    09-03-12
    Location
    London
    Posts
    864
    vCash
    2400
    I've actually put my league patch on hold because this should make it much less time-consuming. Waiting on the edge of my seat.

  24. #18
    Join Date
    15-12-11
    Location
    Your Mother
    Posts
    25,331
    vCash
    4790000
    Don't know if you are taking requests but these 2 are sorely missing from the official editor (and a lot of other tools)

    Filter Free Players
    Filter Free Staff
    Filter Players by Continent
    Filter Staff by Continent

  25. The Following User Says Thank You to Dermotron For This Useful Post:


  26. #19
    Craig Forrest's Avatar
    Craig Forrest is offline Manager
    Official Challenges Mod
    Programmer
    Forum Enforcer
    VIP
    Join Date
    02-03-12
    Location
    The Great White North
    Posts
    4,204
    vCash
    1547
    Filter Stadia by Country..... it can only be done by City and it's a huge pain when making patches..... maybe just for me though

  27. The Following 4 Users Say Thank You to Craig Forrest For This Useful Post:


  28. #20
    Join Date
    15-12-11
    Location
    Your Mother
    Posts
    25,331
    vCash
    4790000
    Nope, that's a good one CF

  29. The Following User Says Thank You to Dermotron For This Useful Post:


  30. #21
    Join Date
    05-06-12
    Posts
    1,035
    vCash
    3700
    @archibalduk - just in case you don't know already. If you ever try opening the original 3.9.68 data you might notice that staff.dat is in a different format; staff preferences are stored in the person data rather than as a 4th 'file'. When I first found this out it freaked me out as my code just stopped working after I had re-installed CM. Running the data through the editor once will modify staff.dat to be in 4 files again.

  31. #22
    Join Date
    14-06-14
    Location
    the English Channel
    Posts
    212
    vCash
    0
    Quote Originally Posted by Dermotron View Post
    Don't know if you are taking requests but these 2 are sorely missing from the official editor (and a lot of other tools)

    Filter Free Players
    Filter Free Staff
    Filter Players by Continent
    Filter Staff by Continent
    Quote Originally Posted by Craig Forrest View Post
    Filter Stadia by Country..... it can only be done by City and it's a huge pain when making patches..... maybe just for me though
    Do you mean being able to look at the data in a spreadsheet and being able to filter by that data? Assuming that's the case:

    Filter Free Players - ALREADY ADDED
    Filter Free Staff - ALREADY ADDED (You can also filter by job for club and job for nation, league contracted, etc etc)
    Filter Players by Continent - I CAN ADD THIS, NO PROBLEM (IT'LL TAKE ABOUT 5 MINUTES' WORK)
    Filter Staff by Continent - I CAN ADD THIS, NO PROBLEM (IT'LL TAKE ABOUT 5 MINUTES' WORK)

    Filter Stadia by Country - ALREADY ADDED (YOU CAN ALSO FILTER BY CONTINENT)

    Here are some screenshots of the exported xml files (the csv files are the same but they don't have the nicely formatted column widths and coloured frozen pane header row):





    Note that the exported_staff spreadsheet is a work in progress (I still need to pre-set the column widths). Owing to the huge amount of data (over 100,000 players and staff), the resultant xml file is about 230mb (and it takes a whopping 17 seconds to generate - whereas exporting city/club/stadium/etc data takes a fraction of a second). The csv file will be smaller, but it's still a bit of a pain. I think I'll remove the staff attributes from the exported_staff spreadsheet and instead export these with the player and non-player attributes spreadsheets.

  32. The Following 2 Users Say Thank You to archibalduk For This Useful Post:


  33. #23
    Join Date
    14-06-14
    Location
    the English Channel
    Posts
    212
    vCash
    0
    Quote Originally Posted by JohnLocke View Post
    @archibalduk - just in case you don't know already. If you ever try opening the original 3.9.68 data you might notice that staff.dat is in a different format; staff preferences are stored in the person data rather than as a 4th 'file'. When I first found this out it freaked me out as my code just stopped working after I had re-installed CM. Running the data through the editor once will modify staff.dat to be in 4 files again.
    Thanks for pointing that out. I hadn't realised that (I've been working with the March roster update).

    I'll proceed on the basis of only supporting the staff.dat with 4 "sub-files" for the time being. Is the Version number different in the index.dat file where the preferences data is stored in the person data?

    Something that bothers me about index.dat is that if your index file is ever corrupt, the staff.dat file is essentially lost because you need the offsets in order to index the 4 "sub-files". I don't like relying on index.dat, instead I prefer just getting the Updater to figure out the index on the fly (i.e. by looking at the file sizes of each file and figuring out the index itself). I think it should be fairly straightforward to calculate the offsets of where each sub-file begins from reading staff.dat. Then I'll be free from having to rely on index.dat.

  34. #24
    Join Date
    15-12-11
    Location
    Your Mother
    Posts
    25,331
    vCash
    4790000
    Nice job and cheers for the additions. So far the Export options look good, any luck with re-importing or have you got that far?

  35. #25
    Join Date
    05-06-12
    Posts
    1,035
    vCash
    3700
    That the solution I've gone for. I don't recall checking the version number on the original data but I expect it will be different (1 instead of 2 presumably).

    My code relies on index.dat - even to the extent of using the file names (staff.dat, club.dat etc). Also allows data to be 'ignored', e.g. one of my tools edits index.dat by making it think there is only 1 staff history - which makes the game load quicker (at the expense of showing the histories in game).

    Not sure how you'd work out numbers based on file size but you could manually scan the file, e.g. read 4 bytes skip 106 bytes, read 4 bytes, skip 106 bytes - then when the last 4 byte number is not 1 above the previous 4 byte number you know that you've got to the end of the person section.

Page 1 of 11 12345 ... LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •