[WIP][TOOL] NeoScavHelperTool

47 posts / 0 new
Last post
#1
[WIP][TOOL] NeoScavHelperTool

NeoScavHelperTool is still unfinished but its purpose, for now, is to work as a viewer for in-game items (GitHub page). Tell me your opinions and issues.

NeoScavHelperTool.zip 14/01/2019

Current working screens:
BarterHexes Viewer
ForbiddenHexes Viewer
HexTypes Viewer
Images Viewer
ItemTypes Beta Viewer
Maps Viewer
TreasureTable Viewer

Spoiler: Highlight to view
IMAGE(https://i.imgur.com/ikMYMH9.png)

IMPORTANT: It will need to have valid XML files on all mods or else an exception will be thrown. List of XML fixes I had to make on some mods:

Spoiler: Highlight to view
*DevKit\BBCAddOn
- Line 1494: escaped '&' to '&'
- Line 2574: escaped '&' to '&'
- Line 2970: escaped '&' to '&'
- Line 3672: escaped '&' to '&'
- Line 3870: escaped '&' to '&'

*DevKit\DevKitAddOn
- Line 253: escaped '&' to '&'
- Line 344: escaped '&' to '&'
- Line 367: escaped '&' to '&'
- Line 390: escaped '&' to '&'
- Line 495: removed extra '>' on the closing tag

*M(m)MoD\M(m)MoD
- Line 20008: escaped '&' to '&'
- Line 20047: escaped '&' to '&'

*Overhaul\AddOn
- Line 2124: removed extra '24031' after closing tag

*Shouldered\Patch_MmMoD
- Line 39: escaped '&' to '&'
- Line 78: escaped '&' to '&'
- Line 117: escaped '&' to '&'

Was this an inside joke? :)

Spoiler: Highlight to view
IMAGE(https://i.imgur.com/sIAyT4k.png)

The tool sounds like a great idea

Thanks for the feedback, there's still a lot of work to do though. Hopefully, I will have some free time in the next weeks to implement more functionalities.

hey, i'm learning programming myself so this peaks my interest a little bit. I'm learning c# and am currently working on a console app that will automatically update any mods ID Numbering to my convention listed here; https://bluebottlegames.com/content/mod-authors-making-better-tomorrow

and what's got my attention is that you said "in-game", which if i'm not wrong means that you must know a decent amount of whatever the games main language is to pull that off right?

OverHaul Mod
DevKit is an upgraded BBC mod.
Improve your mods.

I am sorry when I said in-game I meant that you will be able to view things like they are shown/draw in-game as close as possible (you can check that now, just execute the app and test it). This project is also C# but a WPF one using SQLite as a data store, although I have some experience with GUI-less C/C++ apps I am also learning as I go with this one. For now, my intent is to make a good viewer so anyone can explore all game content including mods. Afterward, my final goal is to also make a good editor so people can easily make new mods or fix things without editing any XML file but this will take a lot of time since I am not able to dedicate the amount of time I would like. If you or anyone have interest in the project and think that can help feel free to reach me.

i can't try to help with it until i finish this one. might take a few weeks as i'm still learning much as well, but it depends on if i can stay on it till complete.

but ya, i can totally envision a simple app that auto-combines vanilla overrides and that auto highlights conflicts. both would absolutely speed up merging and increase accessibility. i have no experience/anything in databases or the SQL stuff. a touch of reading has been on on LINQ.

OverHaul Mod
DevKit is an upgraded BBC mod.
Improve your mods.

when i get a really simple, but working sample i'll send you a link.

OverHaul Mod
DevKit is an upgraded BBC mod.
Improve your mods.

In my code I am already combining all overrides vanilla and mods, it seems that I use a different approach on the merging than the one used by the author of the game but the end result will be the same if the mods are well built (not the case with training mod since it have an issue). The conflicts I wanted to show them on the editor part because they don't make sense in the viewer since they were already merged. I still didn't quite got what you are trying to do but later on, when I get out of work, I will read your post to try to understand. Nevertheless, keep going :)

I got it now, I think you will be able to do something about your advice for "Categorize your IDs;" since that can be automated with relative ease, but about the "Sort your Items types" that will be more tricky since you need to understand in the code the concept of the item type which will be very tricky. Nevertheless, I think it can be somewhat done using the "vProperties" of the item to categorize it. By the way, nice work doing these conventions, at least on the items it seems the one used by the game author. In the far future when the editor is being coded I will try to generate the XML files using these conventions.

It's 100% possible to automate the entire ID numbering process (but certainly not the "name your code blocks with a tag" part) because i already did the legwork on a editor called NotePadPro.

On that editor, i created entire RegEx that finds and replaces every ID to the correct catagory/leading type number. i accomplished this by having the editor look at things before and after the number to figure out it's context and to act accordingly, along with some help from the user.

Unfortunately, that editor only allows me to replace one number at a time, and gives me very little tools to automate the entire process. so i'm recreating the entire thing in c# now, which should allow me to do the entire job in 1 button press with more efficiency and accuracy. i already did a mockup program of it to see what it might look like when done, and i'm creating the core of the code in a separate and more platform universal library (.net standard 2.0, which is looking pretty good for cross-platform support which .net standard to use) so that I can put this on Linux and Mac (which i have no idea how to use, but hopefully i'm laying the ground-work)

so what i'm saying is, when polished out you should be able to use my library to implement it to your program.

OverHaul Mod
DevKit is an upgraded BBC mod.
Improve your mods.

Yes, this would be re-usable even if you don't make a library we can always just fetch the code into this project when needed.

hey this makes me think you already have a modified xsd of the games xml? can i have a copy please

OverHaul Mod
DevKit is an upgraded BBC mod.
Improve your mods.

As a matter of fact, I don't use any xsd file, I did a XML SAX parser approach and use the table element to select the database table destination and next read all values and columns with a key-value pair approach and insert them into the database. Nevertheless, check on Github the DBOperations.cs file and ParseXMLFile method if you want.

hey wanna get on the games Discord channel and talk sometime? i'd like to see a short video showcasing what your app does so far. the game has an official discord.

OverHaul Mod
DevKit is an upgraded BBC mod.
Improve your mods.

I've never used Discord but I will give it a try when I feel that the app is more or less refined and useful. Right now the app is really basic and doesn't do much. Also, I am doing some code re-write, since this is my first approach to WPF I learned that I didn't really have done good and maintainable code and so, now I have even fewer functionalities but I will be working on it this week since I am on vacations. Happy new year :)

Just added some screenshots of what is working for now if you're curious.

seems like some great potential here. i don't think i could make a WPF yet, so far i only feel comfotible making simple black box console apps.

is the editor a full on text editor? and will the viewer let me "click" on say, a battle move and see everything that uses that block of code, and every block of code that battletype uses? like a giant web so i can see how everything's connected.

OverHaul Mod
DevKit is an upgraded BBC mod.
Improve your mods.

WPF is somewhat tricky but I also had neither used windows forms or WPF before and it seemed a more modern approach and simpler than windows forms at first glance. That was the reason for my choice.

The editor will be just like the viewer but with edition capabilities using the GUI, then I will generate the XML files internally. I hope I will be able to make it like I am planning, but for now is just a placeholder, doesn't do anything and will not do anything for a long time (first I want to finish the viewer completely).

The viewer first I want to make basic info and renderer display for all database table items, so no giant web at this point, unless one understands the database variables and choose the relation item from the tree manually. When it reaches this point, the viewer should already be useful for a lot of people. Nevertheless afterward, my intent is to make advance info and renderer display for the database table items, I still don't know for certain what that advance means since I didn't give it much thought yet but, at least I am thinking on doing those web relationships at least on a basic level by having buttons on the table instead of just text. Example table column "nTreasureID" instead of being just a number "149" it would be a button that if clicked would change the viewer to that treasure item (here I will try to add navigation capabilities (back and forward buttons) if I can).

Ok, so just looking at the image i have a few ideas for plugging my Naming Convention into your program.

I believe i heard, that these GUI forms you're making can open up a black console window and run my console program as if it was a console program.

If that's true, i think combining my program with yours when it's done would be relatively simple. So here's what i'm thinking:

Up-top you have a viewer and editor tab, just add another tab called "ConventionImplementor" or something...not sure on the name or presentation yet...but something for me so that while you work on your program, i can download it from github and fill in that tab.

the idea is that with that tab, i should be able to combine my program with yours without interfering with your work. Users will click on the tab, and then hit a start button or something. this button will open up a black console box, and do everything else as if they had started my standalone version.

i can worry about filling in the details of how the GUI looks when they click on that tab until you're ready to focus on that area, but i imagine that, all i need on it is a button that merely runs the console window.

it's not anything we have to do right now as my program is still in it's infancy.

OverHaul Mod
DevKit is an upgraded BBC mod.
Improve your mods.

Yeah, that sounds reasonable for integrating on the GUI, but I would prefer to call a dll or if you don't mind to have your code open-source, we can just copy your source code files into the project. Nevertheless, calling another process is doable. As you said, later on, we will worry about that.

ya it's just so far i have alot of "Console.WriteLine();" methods in my .dll so i figured that'd be the easier way.

i did it that way just because i didn't want to think about it, but in the end i might try to replace all the writelines with a string variable and have the application in question need to use them. but it interferes with the flow of how my methods run so it creates more work. maybe when i get a complete sample ready you can look over it with me and we can talk about how i can do this better.

i don't mind making it open source, there just isn't anything worth posting yet for github.

OverHaul Mod
DevKit is an upgraded BBC mod.
Improve your mods.

Sure I would help you look into the code and try to understand the best way to integrate but as I said it is possible to run another process and so we can do this on a first step and later improve on that, so don't worry about it, just code normally :)

ok so, something that's getting to me,

how will you handle mal-formed xml files a user inputs into your program?

like that's a really big question. on my, i decided not to handle it. I just use RegEx to find the numbers, add some leading numbers to them, and done. because i'm using regex to do everything, i don't have to worry about the system complaining that the user has a missing --> or an & with no & in their xml.

this does mean i have to do some extra work with the RegEx though, but i know that 90% of the people using my program barely know what a computer is.

i would like it if i could just expect them to clean up their xml...so that makes me ask, did you already solve this problem? or do you already have a plan for this?

like is there an app out there that walks noobs through the process and we can just link them the app? or have you not touched that area yet and are just planning on touching it when you hammer out the rest of the program?

i ask because if you already have that part done then maybe i'll abandon ship on the console app and just go straight to building it on your app instead.

OverHaul Mod
DevKit is an upgraded BBC mod.
Improve your mods.

Right now I also don't handle malformed XML, with the exception of comments within comments, because I filter the input files with a regex to remove that. I think that 90% of time XML will be valid and that was the reason I didn't worry too much at this point, but if this reveals to be a problem we can always make a custom XML parser that would validate the files first and pinpoint the problems. Although, I said 90% currently some mods have malformed XML which I had to fix and listed the changes in a spoiler on this thread.

Off topic:
Do you have knowledge if this is valid or not? (xml is valid, i mean game valid)
"<column name="vEquipSlots">13=ShoulderMod:0=ShoulderMod:0,14=0=ShoulderMod:0,21=2=2,20=2=2</column>"
I am kind of struggling to understand/find documentation so I know what I can expect to find while parsing.

it's been awhile since i looked into vEquipSlot syntax, but it could be right.

Quote
ItemSlot Chart

vEquipSlots -Where you can equip this item eg.(20=0,21=0,22=0) The "=0" is optional and denotes which image from vImageList (0-based index) should be used when the item is in each slot.

OverHaul Mod
DevKit is an upgraded BBC mod.
Improve your mods.

Yeah, and these indexes takes precedence over the vImageUsage ones, that I got it. My doubts are with the "ShoulderMod:0" specifically, shouldn't it just be the index number since the image itself is the one which has the address of the mod to fetch it? Nevertheless, I will try to spawn the item in-game on the start treasures to check if all interactions are correct. I have done a beta itemtypes preview and this item is the only one where it crashes since I was not expecting that syntax but if the game handles it I will handle it with ease also.

Also, thank you for that OneNote Online documentation, didn't know the existence of that one until now and there are the slots information that I was needing :)

the OneNote was a project i started, but it's on hold for an unknown amount of time. i can add you to the list of those who can edit it if you wanna contribute. the others seem to have lost interest in updating it.

OverHaul Mod
DevKit is an upgraded BBC mod.
Improve your mods.

Sure, eventually I will need to understand all the properties in tables and their format to be able to code correctly.

ill need an email

OverHaul Mod
DevKit is an upgraded BBC mod.
Improve your mods.

Just sent you a dm with it. After I tested the above item in-game I confirmed that indeed it was a correct format and so I coded it. I'm about to update the app here with the new screens.

indeed, this will make a difference i think.

i think someone started something sooorta like this on the forums before, but abandoned it long ago.

hey, you might want to put all those photo's on one vertically long canvas so that i can full screen it and just scroll down to view each one.

The program is looking like a girls gone wild event, and due to some mistake, this program is the only guy present. good times.

OverHaul Mod
DevKit is an upgraded BBC mod.
Improve your mods.

OverHaul Mod
DevKit is an upgraded BBC mod.
Improve your mods.

Hey, looking good. About the problem of checking if it's vanilla override or not and the input of XML files, why don't you parse the getmods.php and only ask for this file? This way you have access to all XML files at once.

Edit: followed your advice and merged images vertically.

I suppose that is the better, and simpler solution. I originally didn't take that route because perhaps they don't want one of the files "adjusted".

However, that shouldn't be a problem because i'll eventually add logic to make sure the ID that are about to be adjusted, aren't already following convention. Probably by checking the length of the ID (my convention has it so that ID's are a minimal of 5 digits long, using zeros if need be) and if those digits already have the correct leading number. so if the ID already matches these two criteria it's likely that this ID shouldn't be touched.

I suppose, since the logic is already there, i'll keep the current solution in place, but offer to the user if they want to use the .php or manual input instead. Or maybe i wont, and just use php only so i don't have to manage the extra code.

OverHaul Mod
DevKit is an upgraded BBC mod.
Improve your mods.

The problem of not using the getmods.php is when mods override other mods, the IDs will not be consistent until the user runs on all XML files I think. Nevertheless, users that will run your app should be mostly users with modding understanding and so should be able to solve problems.

Just added the treasure table screen to the app which let me noticed some game syntax problems in the XML of on some mods but I will not fix this for now. First I will add the other basic screens and then make a validator to help fix the problems.

looking hot, question: what if i just want a scrollbar on the image part of the treasure table? this way, it's easier for me to visualize where it's at on the Line of code. i mean it'll work as is, this will almost make my naming convention unnecessary. (except that not everyone will use the program, and even if they did, they may not use it all the time)

OverHaul Mod
DevKit is an upgraded BBC mod.
Improve your mods.

Not sure if I understood correctly your question, but you meant like the HexTypes view where I have a separator between the tile image and the table, which, basically divides the right side of the screen in two? Also, right now I am showing just the table representation recursively (hopefully there are no loop references since I didn't code a protection) but later on I will add another way to show only the final items and their final computed probability to occur. Now that I have implemented this I can move on to recipes which I think that can be really useful, not sure if I need the ingredients first thought. Your modding conventions are nice, they will help a lot when analyzing the XML files, my program will not be useful for that in a long time.

my program inadvertently helped me find an error i didn't know about in my overhaul mod.
IMAGE(https://i.imgur.com/WOajxzm.png)

i don't have any attackmode ID's, but i think this is a relic from an idea i scrapped awhile ago.

i think, in your program, we should add a feature that checks that every ID used actually has an ID to be used, and that each ID is indeed used. if such an inconsistency is found, it gives a warning. make it another tab on your WPF called "error check" or something.

Ok, i decided I'm going to start working on an XML schema for both of us. This will help everyone, as it'll make catching mistakes much easier and ensure that are programs have the right things to work with. more effort will be required from mod authors, but they always needed that anyway and honestly it shouldn't take much to learn that they can't just mash buttons on a keyboard and expect things to work. i think the schema will let us easily spot when someone misspelled a node or left out a slash or angled bracket because they don't see the problem with manually typing out each element.

then with that, maybe i can find an app or something that we can link to that not only validates that but also has a decent debugger that these people can use so that we don't have to create our own xml fix thingy. may have found such a thing here: [url]https://www.xmlvalidation.com/[/url but i haven't looked into it much yet.]

OverHaul Mod
DevKit is an upgraded BBC mod.
Improve your mods.

Schema created.

Because attributes and elements do NOT have any semblence of a unique name for each one...there was plenty of opportunity lost.

however, this schema has plenty to offer still.

XSD

it offers a list of table table types and children that are allowed to exist, meaning if you accidently mispell the name of a tag, attribute, or the value of one of those, then it'll catch the error and point you to it.

it'll also spot things like this:
IMAGE(https://i.imgur.com/ereXKh1.png)

next, i'll get to work with having my library explain why a Well-Formed check failed to the user, and then adding the schema on top of that, and then laying some RegEx on top of that as a third layer to do some things the xsd was not able to pull off. mostly, just error checking, but more for the user at that point than for our program.

p.s.
OH ya, i ran this on my Overhaul and Devkit, and fixed things up in both. so you might want to download the updated versions of those.

OverHaul Mod
DevKit is an upgraded BBC mod.
Improve your mods.

Haven't "added" anything a user will notice since the video, but doing a major refactoring now that the schema is done. Focusing more on multiple libraries just to give it a try, and this time ima include your project from the ground up since i'm using delegates to accomplish the messages i was delivering.

OverHaul Mod
DevKit is an upgraded BBC mod.
Improve your mods.

ok i downloaded your github thing, and we should get on discord because i'm new to alot of of stuff and i'm curious as to how to have multiple custom made libraries in one project, and how two people might work on a project together.

OverHaul Mod
DevKit is an upgraded BBC mod.
Improve your mods.

Yes, refactor is a major part of coding when we start we never understand the problem completely and after some "prototypes" a refactor in code is always needed so the code seems more right (it never will be 100% right :p). Ok, sure I can go online more or less 2h from now (I'm still working). Although you can start by forking my code to your GitHub, afterward, you get a git client (I prefer with GUI) and check out the code from your GitHub and then create a branch and work normally. Later if I catch you online I will explain with more detail if needed.

well i already downloaded your git files, and my first goal is to create a tab right next to Editor that host my functionality.

when we get that out of the way, any changes i make shouldn't have any impact on what you're doing then.

but come join the games official Discord channel, and if we see each other on just @Mention me.

https://discord.gg/GQJPdw9

OverHaul Mod
DevKit is an upgraded BBC mod.
Improve your mods.

i don't know if you're checking your Discord so i'll post this here instead:

suggestion, instead of locking a user to the root getmods.php, instead let them choose the php file after the vanilla files are other constants are loaded in?

truth is, i don't like that the root getmods is what's selected because it means i must install, merge, and prepare the files each time I want to switch mod sets. i'd rather just have the mods in place, but drag and drop in the getmods and go from there.

maybe you just haven't gotten that far yet and already planned to do it that way

OverHaul Mod
DevKit is an upgraded BBC mod.
Improve your mods.

If I understood correctly you were, suggesting to have something like a mod manager, wasn't it? That is doable, I thought about that before because I've downloaded an executable file somewhere here in the forum that does that at a command line level and it could go into the GUI but I wasn't planning on doing it right now, unless if you feel that it is more urgently than do the remaining of table data visualization.

no it's not urgent.

OverHaul Mod
DevKit is an upgraded BBC mod.
Improve your mods.