Page 2 of 11 FirstFirst 123456 ... LastLast
Results 26 to 50 of 267

Thread: CM01/02 Updater (a DB/Pre-Game Editor)

  1. #26
    Join Date
    04-03-12
    Location
    Serbia,Belgrade,Zemun
    Posts
    937
    vCash
    900
    looks great...spreadsheet is almost exact as spreadsheet i used to insert data via JL uber tool...
    how fast will be inserting option?just for comparing,JL uber tool can insert around 5000 staff with full attributes under 10 minutes...
    just suggestion:

    option for adding clubs,cities,stadiums to DB via spreadsheet (same as JL tool Uber Parser does)
    option for altering club and nation histories via spreadsheet

    Also...there is a handy option in Uber Parser tool (update club option) through .csv file that allows u to relocate great amount of clubs at once from league to league (specially great for league patching),so i wonder if something like that can be included.
    thanks for the great work m8!

  2. #27
    Join Date
    14-06-14
    Location
    the English Channel
    Posts
    217
    vCash
    0
    Quote Originally Posted by Dermotron View Post
    Nice job and cheers for the additions. So far the Export options look good, any luck with re-importing or have you got that far?
    No importing quite yet. I think that might not be until next month. I'm just finishing off data deletion and the free agent utility. Then I'll release it for people to test.

    Regarding data import/filtering - if there is anything else you need, please let me know. Adding data export stuff is a simple 10 minute job.

    Quote Originally Posted by JohnLocke View Post
    That the solution I've gone for. I don't recall checking the version number on the original data but I expect it will be different (1 instead of 2 presumably).

    My code relies on index.dat - even to the extent of using the file names (staff.dat, club.dat etc). Also allows data to be 'ignored', e.g. one of my tools edits index.dat by making it think there is only 1 staff history - which makes the game load quicker (at the expense of showing the histories in game).

    Not sure how you'd work out numbers based on file size but you could manually scan the file, e.g. read 4 bytes skip 106 bytes, read 4 bytes, skip 106 bytes - then when the last 4 byte number is not 1 above the previous 4 byte number you know that you've got to the end of the person section.
    Yeah I used to rely on index.dat very heavily too. However, I now load the full DB each time into individual classes, so I can't rely on index.dat any more.

    Regarding the staff files - It would be a case of tracking the ID numbers of each entry read until you reach 0 again (at which point you know the next sub-file begins). I think it's a bit of overkill, but maybe I'll look at it once other more important things are done.

  3. #28
    Join Date
    14-06-14
    Location
    the English Channel
    Posts
    217
    vCash
    0
    Quote Originally Posted by djole2mcloud View Post
    how fast will be inserting option?just for comparing,JL uber tool can insert around 5000 staff with full attributes under 10 minutes...
    I can't remember the exact numbers, but IIRC the EHM Updater can insert around 100,000 player career history entries in under a minute (the EHM career history entries are *slightly* larger than the CM history structure, so I'd hope the CM Updater to be marginally quicker).

    Adding and updating data via spreadsheets will come with time. Player, non-player and official attributes, staff data (e.g. contract data) updating/editing and player/non-player and player career history insertion are my first tasks. But I can't commit to any ETA I'm afraid.

  4. The Following User Says Thank You to archibalduk For This Useful Post:


  5. #29
    Join Date
    05-06-12
    Posts
    1,035
    vCash
    3700
    Just out of curiosity, what programming language do you use?

  6. #30
    Join Date
    14-06-14
    Location
    the English Channel
    Posts
    217
    vCash
    0
    I use C++ and the Qt Library (Qt makes GUI programming much easier than pure C++).

  7. The Following User Says Thank You to archibalduk For This Useful Post:


  8. #31
    Join Date
    05-06-12
    Posts
    1,035
    vCash
    3700
    If you're interested, here are a couple of features I have planned to implement in my CSV tool:

    Attributes can be set using a 5-star rating system as well as being set explicitly, e.g. you can enter '*' to '****** which will convert a random number as follows: 1-5, 4-9, 10-13, 14-17, 18-20. I have designated dirtyness and injury proneness as 'bad' attributes - so if you enter ***** you'll get a low value rather than a high.

    For club kit colours I'm planning to allow the user to use the name of another club as well as the colour names, e.g. it would accept 'Man Utd:away' then copy the values from Man Utd's away kit.

  9. The Following 3 Users Say Thank You to JohnLocke For This Useful Post:


  10. #32
    Join Date
    02-03-12
    Posts
    4,850
    vCash
    1075
    archibald, can you also add the option to export stadiums attached to clubs that use them?

    i mean extracting a list of stadiums like this:

    etihad - man city
    emirates - arsenal
    anfield - liverpool

    i know it's not so useful maybe but as i'm a maniac of tables, it would save me some work before every release

  11. The Following User Says Thank You to milo For This Useful Post:


  12. #33
    Join Date
    14-06-14
    Location
    the English Channel
    Posts
    217
    vCash
    0
    Quote Originally Posted by JohnLocke View Post
    If you're interested, here are a couple of features I have planned to implement in my CSV tool:

    Attributes can be set using a 5-star rating system as well as being set explicitly, e.g. you can enter '*' to '****** which will convert a random number as follows: 1-5, 4-9, 10-13, 14-17, 18-20. I have designated dirtyness and injury proneness as 'bad' attributes - so if you enter ***** you'll get a low value rather than a high.

    For club kit colours I'm planning to allow the user to use the name of another club as well as the colour names, e.g. it would accept 'Man Utd:away' then copy the values from Man Utd's away kit.
    That's a really neat idea about the team colours. I might "borrow" that idea for my EHM Updater!!

    The rating system is intriguing. Something I'd been thinking about for the EHM Updater was to create some player profiles or roles which weight the attributes according to the player's profile (e.g. a physical attacking winger). I.e. you select the role and then the tool will create an appropriate set of attributes according to this role and the CA.

    Quote Originally Posted by milo View Post
    archibald, can you also add the option to export stadiums attached to clubs that use them?

    i mean extracting a list of stadiums like this:

    etihad - man city
    emirates - arsenal
    anfield - liverpool

    i know it's not so useful maybe but as i'm a maniac of tables, it would save me some work before every release
    Currently it lists all of the clubs and says which stadium is attached to each club. Is this what you mean, or are you looking for a spreadsheet that lists every stadium and says how many teams play at that stadium?

  13. The Following 2 Users Say Thank You to archibalduk For This Useful Post:


  14. #34
    Join Date
    02-03-12
    Posts
    4,850
    vCash
    1075
    Quote Originally Posted by archibalduk View Post
    Currently it lists all of the clubs and says which stadium is attached to each club. Is this what you mean, or are you looking for a spreadsheet that lists every stadium and says how many teams play at that stadium?
    the second one

    i need it to know which stadiums are not attached, not needed.

    currently i extract the list of stadiums, compare it to the list of clubs with their stadiums deleting those used by clubs, then i compare it again to the list of nations with their stadiums deleting those used by national teams.

    what remains is what i was looking for, the list of unattached stadiums.

    same for cities and related stadiums...

  15. #35
    Join Date
    14-06-14
    Location
    the English Channel
    Posts
    217
    vCash
    0
    Ok I see. Leave it with me. I'll see what I can do!

  16. The Following User Says Thank You to archibalduk For This Useful Post:


  17. #36
    Join Date
    14-06-14
    Location
    the English Channel
    Posts
    217
    vCash
    0
    Quote Originally Posted by milo View Post
    the second one

    i need it to know which stadiums are not attached, not needed.

    currently i extract the list of stadiums, compare it to the list of clubs with their stadiums deleting those used by clubs, then i compare it again to the list of nations with their stadiums deleting those used by national teams.

    what remains is what i was looking for, the list of unattached stadiums.

    same for cities and related stadiums...
    Is this what you had in mind?



    This lists all clubs and nations attached to each stadium. It also lists blank stadia (just use the Data -> Sort or Auto-Filter in Excel/etc to filter them out). There are 2,037 unattached stadia (out of 9,490 stadia) in the March DB update.

    What is the deal with the stadium named "Player and Comp History File Club"? Is it a game-critical stadium? It has 316 stadia attached to it.

    Let me know if you need anything else displayed on this spreadsheet.

    Assuming this is what you had in mind, I'll get started on the stadia by cities spreadsheet later this week/weekend.

  18. The Following 3 Users Say Thank You to archibalduk For This Useful Post:


  19. #37
    Join Date
    02-03-12
    Posts
    4,850
    vCash
    1075
    Quote Originally Posted by archibalduk View Post
    Is this what you had in mind?



    This lists all clubs and nations attached to each stadium. It also lists blank stadia (just use the Data -> Sort or Auto-Filter in Excel/etc to filter them out). There are 2,037 unattached stadia (out of 9,490 stadia) in the March DB update.

    What is the deal with the stadium named "Player and Comp History File Club"? Is it a game-critical stadium? It has 316 stadia attached to it.

    Let me know if you need anything else displayed on this spreadsheet.

    Assuming this is what you had in mind, I'll get started on the stadia by cities spreadsheet later this week/weekend.
    2037!?!?!?

    i was sure i deleted them all in the previous release... or maybe i deleted unattached cities... can't remember now

    yes archibald, it's just what i have been wishing for, thank you so much

    the "player and comp..." stadium is a fictional stadium that help us not to delete clubs needed for history files, basically we set that stadium to extinct clubs in db so that if you wanted to delete the club you can see you don't have to

  20. The Following User Says Thank You to milo For This Useful Post:


  21. #38
    Join Date
    14-06-14
    Location
    the English Channel
    Posts
    217
    vCash
    0
    I've finished the spreadsheet showing stadia grouped by city. Melbourne has the most stadia attached (64 stadia). 496 of 7229 cities don't have a stadium attached.

  22. The Following User Says Thank You to archibalduk For This Useful Post:


  23. #39
    Join Date
    14-06-14
    Location
    the English Channel
    Posts
    217
    vCash
    0
    I've been making a little more progress with the Updater this weekend. The lasted feature I've transferred across from the EHM Updater is a screen which shows some stats about the Current Ability ratings of players in each league. It shows stats such as the league average as well as how many players have a zero CA:



    I'm hoping to have a version online within the next couple of days. I had hoped to have it uploaded today, but the Delete Staff function is still buggy. It might be that I won't have the Delete Staff function working before I release it (I need to take some time off from working on the Updater for a week or two to work on some other things).

  24. The Following 9 Users Say Thank You to archibalduk For This Useful Post:


  25. #40
    Join Date
    15-10-12
    Location
    Grantham
    Posts
    371
    vCash
    0
    Quote Originally Posted by archibalduk View Post
    I've finished the spreadsheet showing stadia grouped by city. Melbourne has the most stadia attached (64 stadia). 496 of 7229 cities don't have a stadium attached.
    Just a thought as it was something I could have done with a few months ago when I was looking at stadiums. Not just viewing stadiums by city, but also by country. Is this possible?

  26. The Following 3 Users Say Thank You to Golly For This Useful Post:


  27. #41
    Join Date
    14-06-14
    Location
    the English Channel
    Posts
    217
    vCash
    0
    Quote Originally Posted by Golly View Post
    Just a thought as it was something I could have done with a few months ago when I was looking at stadiums. Not just viewing stadiums by city, but also by country. Is this possible?
    Yeah, that's not a problem. I'll add a Nation column to the stadia spreadsheets.

  28. The Following 2 Users Say Thank You to archibalduk For This Useful Post:


  29. #42
    Join Date
    15-10-12
    Location
    Grantham
    Posts
    371
    vCash
    0
    Quote Originally Posted by archibalduk View Post
    Yeah, that's not a problem. I'll add a Nation column to the stadia spreadsheets.
    Oooh I love you. I already have a son called Archie, but if I didn't I would name any further son I might have in your honour.

  30. The Following 2 Users Say Thank You to Golly For This Useful Post:


  31. #43
    Join Date
    14-06-14
    Location
    the English Channel
    Posts
    217
    vCash
    0
    Quote Originally Posted by Golly View Post
    Oooh I love you. I already have a son called Archie, but if I didn't I would name any further son I might have in your honour.
    Haha that's quite a compliment!

    Give me a shout if there's anything else you'd like to export from the DB in spreadsheet form. It's pretty straightforward to add exporting functions.

  32. The Following 2 Users Say Thank You to archibalduk For This Useful Post:


  33. #44
    Jesus is offline Official Challenge Team
    Social Media Team
    Join Date
    04-03-12
    Posts
    3,534
    vCash
    681
    This is such an amazing project, hats off to you Archi.

  34. The Following User Says Thank You to Jesus For This Useful Post:


  35. #45
    Join Date
    14-06-14
    Location
    the English Channel
    Posts
    217
    vCash
    0
    As promised, here is version 0.0.1 of the CM 01/02 Updater: http://1drv.ms/1oNV0aY

    Please note that the Updater is still in a beta state. The Updater has not yet been fully tested. I'm uploading it now because I don't think I'll have much time to do more work on the Updater for a week or two. If anybody tests it out then please let me know what does or does not work (exporting should work fine, but the other functions may be buggy - I know for certain that parts of the Erase DB function is buggy - see more details below).

    It is imperative that you backup your database before editing it using the Updater. You do not need to backup your database if you just use the exporting functions and don't click on File->Save afterwards. It's only if you're going to use the File->Save function that you need to first backup your database. Alternatively, you can use File->Save As to save the database in an different location.

    Here are some notes:


    Minimum Requirements
    You must have the Microsoft Visual C++ 2010 Redistributable Package installed otherwise you'll encounter a crash when you try to load a database in the Updater. Here's the download link: http://www.microsoft.com/en-us/downl...s.aspx?id=5555


    Installation
    Simply extract the contents of the downloaded zip file into a folder of your choice. You'll see that the zip file consists of the exe and a number of dll files (including a 'platforms' sub-folder which consists of 3 further dll files).

    If you're going to edit and save a database located in C:/Program Files/../ then you must use 'Run as Admin' to load the Updater - otherwise it won't be able to save your database properly (as with any other editor, if you don't use Run as Admin for a database in Program Files, you'll end up with a shadow copy in your /Users/Roaming/ folder which is a real pain). Also, you must always use 'Run as Admin' if you install the Updater to C:/Program Files/../ (otherwise it won't be able to save your settings to its config file).

    I have Windows 7 on my laptop and so this is what I've tested the Updater on. However, I keep an old XP installation via a virtual machine and have briefly tested the Updater on XP (it seems to run ok).


    CSV and XML Spreadsheets
    You can export data to either CSV or XML file formats. You'll see in the status bar (bottom right corner of the window) that it says 'CSV' or 'XML' depending on your current setting. Click on Spreadsheet->Use CSV Spreadsheets if you want to use CSV rather than XML (and click on it again to re-enable XML). Also, when you click on the Export button you'll see from the Save File dialogue window that you can choose the file extension regardless of your CSV/XML setting. The CSV/XML setting is just a quick way of setting the default.

    The advantages with CSV files are:
    - They're smaller in file size than XML;
    - They are fully compatible with OpenOffice/LibreOffice as well as Excel.

    The advantages with XML files are:
    - They have nicer formatting by default which the Updater utilises - such as frozen header panes and pre-set column widths;
    - They are usually compatible with OpenOffice/LibreOffice. The only time these programmes struggle is if you load particularly large XML files (such as the exported staff and staff history spreadsheets);
    - They are full compatible with Excel. Unlike OpenOffice/LibreOffice, Excel doesn't seem to have any problem with large XML files;
    - They fully support UTF (whereas CSV is better suited to ASCII - not even Excel seems to particularly like UTF CSV files - at least from what I've tested). I don't think this is really an issue for CM because I haven't come across any UTF characters thus far (unlike EHM);
    - Unlike CSV files, they don't have different locale/regional types (see below for more details).

    A particular drawback with CSV files is that the delimiter varies according to your computer's locale (or regional settings). A CSV file is simply a text file with a set delimiter (or separator) which acts as a column/cell separator. Computers with a British, Canadian or US locale use commas as delimiters. Most other countries use a semi-colon as a delimiter (because those countries use a comma as a decimal point). There is also a third option which OpenOffice/LibreOffice often uses by default - tab delimiters.

    You can see what delimiter your system uses by clicking on the Spreadsheet menu. You'll see at the bottom of the menu, it'll say in grey text "CSV Delimiter: " followed by either "comma" or "semi-colon".

    The various CSV delimiters is not a problem for the Updater because it automatically detects your locale and will export using the appropriate delimiter. It will also auto-detect the delimiter of a given file when the Import function is up and running. The difficulty arises when you want to share a CSV file with somebody from a different locale - it won't appear correctly in their Excel/OpenOffice/LibreOffice. To solve this problem, you can use Spreadsheet->Convert CSV Delimiters to convert the delimiters for any CSV file.


    CSV and XML Spreadsheets - Which Should You Use?
    The above section was a bit long-winded, so I thought I'd cut to the chase here: Given the choice, I would go with XML files where you can. The only time you might want to consider using CSV is if you're exporting a particularly large file (e.g. the staff spreadsheet) because when using XML the resultant file will be approx 170 mb (it takes around 15 secs to export). Compared with using CSV where the export file is 15 mb (4 secs to export). To get around the large size of the staff XML file, you could load it in Excel/etc and save it as an XLS/XLSX as they are smaller than XML files.


    Export
    This is no doubt obvious:- Use the drop-down menu to select what you want to export and then press the Export button.

    I haven't had the chance to recently test the Export All option, but I'd expect it to take perhaps around a minute with XML and maybe about 30 secs with CSV.

    Note that I need to tidy up the column headings in the exported attributes spreadsheet (for both players and non-players IIRC).

    I plan to add additional export options in the future. If there is something in particular you'd like to be able to export then please let me know.


    Erase Data
    This function is buggy and still needs more testing and work.

    CM doesn't like having any blank tables. To get around this, the Updater will erase the data and then create a single entry. From memory, this seems to work ok for erasing officials, staff career history and staff competition/awards history (note that when I say "staff" I mean both players and non-players).

    Erasing club competition history doesn't cause a crash but it does cause a couple of errors to pop-up during game creation. I would guess that it's because certain competitions must have at least one entry of history. I'll need to test out this theory.

    Deleting staff causes a crash without any error message. I need to figure this one out. Presently the Updater will create a single staff member who is a player/coach (thereby ensuring that there is a single entry in both the player and non-player tables). I thought this would work (as it does for EHM), but alas it crashes. Hence using the "Create a blank database" option from the drop-down menu also causes a crash (because it involves deleting staff). Hopefully I'll get to the bottom of this (it might be because the Updater doesn't update the staff counts in the nations table yet).


    Free Agent Utility
    Seems to work, but more thorough testing is needed.


    Year Adjustment
    Seems to work, but more thorough testing is needed.


    Current Ability Ranges
    This table shows the range of Current Ability ratings for players contracted to each league. You can export this to a spreadsheet via the Export button below the table (as you can also do so for the Index table).
    Last edited by archibalduk; 18-07-14 at 08:12 PM.

  36. The Following 12 Users Say Thank You to archibalduk For This Useful Post:


  37. #46
    Join Date
    15-12-11
    Location
    Your Mother
    Posts
    25,859
    vCash
    4300000
    Jiminy fucking jilickers


    4 secs and not 40 minutes, beautiful

  38. The Following User Says Thank You to Dermotron For This Useful Post:


  39. #47
    Join Date
    15-12-11
    Location
    Your Mother
    Posts
    25,859
    vCash
    4300000
    Insanely quick, top effort archie

    I would suggest adding the CA/PA/HR/CR/WR to the exporter when you get a chance (and the others stuff I guess). So quick to export

    Wonderful start
    Last edited by Dermotron; 17-07-14 at 10:43 PM.

  40. #48
    Join Date
    15-12-11
    Location
    Your Mother
    Posts
    25,859
    vCash
    4300000
    Apologies, just seen that was missing for Staff, not Players, truly awesome stuff

  41. #49
    Join Date
    14-06-14
    Location
    the English Channel
    Posts
    217
    vCash
    0
    I'm on my mobile right now, but forgot to mention that if the Updater doesn't run then you'll need to install the Microsoft Visual C++ 2010 Redistributable. I'll post a link to it tomorrow.

    Dermotron - my next task is to tidy up the player and non-player attribute spreadsheets with better column titles. IIRC CA/PA/Rep are already listed in those spreadsheets, but they might not be clearly labelled right now. [Edit - I see you've found them now ]

  42. #50
    Join Date
    15-12-11
    Location
    Your Mother
    Posts
    25,859
    vCash
    4300000
    One small change for future usage mate, the history exporter should probably use Club Shortname instead of Longname since thats what the History Tool that put them there uses

    Of course, if you are developing your own History Tool/History Importer that's no big issue really

Page 2 of 11 FirstFirst 123456 ... LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •