MosesofEgypt has contributed to 308 posts out of 463386 total posts
(.07%) in 1,548 days (.20 posts per day).
20 Most recent posts:
Well if anyone is willing to host them, i can send them over skype or discord. It's 824MB with everything compressed. I aint got dat much dropbox room. Also, for anyone wanting the tags, just rip with refinery. That's how i got them. I can pretty much guarantee refinery rips perfectly at this point. Not so sure about OS maps tho, cause i have been missing fields in them in the past and i never tested ripping and recompiling one.
Copy + pasted his list here and sorted it based on what my port fixes. He can certainly improve on mine, but remember, I did this in roughly one day. Proper extraction is the key. If he wants, he's welcome to use Refinery to extract the campaign maps I uploaded, fix the things I didn't(and that issue with the shader_environment tags in fog), and uploaded a "Perfectly Refined" version.
Fixed AI grenade throwing. AI couldnít throw grenades because of corrupt throw velocity values for actor_variants (thanks to various people, such as Masterz1337, for providing info on this, as well as estimate values for the velocities).
Addressed Sentinel shields so they arenít constantly visible on Sentinels that donít have shields at all. Though I think, due to some issue with the modifications Gearbox made to CEís engine, I canít seem to get the shields to appear only when damaged.
Fixed an issue with health packs on A10 (Pillar of Autumn) and D40 (The Maw) that were on the ground when they were suppose to be on the wall.
Fixed some encounters, so they work properly, such as the first encounter on a50 (Truth and Reconciliation).
Fixed several instances of missing multipurpose maps and shaders, such as for the Assault Rifle and Halo ring (thanks to Bungie for the original assets and to various community members for bringing them to Custom Edition).
Fixed several lightning effects, such as those found in C20 (The Library) and the Monitorís lightning tendril.
Attempt to fully fix the sentinel shield problem.
Fix the Longsword missing its multipurpose map.
Pretty sure these are fixed:
Fixed fog in the exterior areas of B40, so that it looks just like what can be found on the Xbox.
Fix issues with fog on several levels.
Fix Cortana not changing colors in Haloís Control Center.
Unsure if fixed:
Fixed a couple of instances of objects having incorrect impact materials.
Fix graphical issues with glass on B40 (Assault on the Control Room) and C40 (Two Betrayals).
Fix the missing shield flare for the Master Chiefís shields when recharging.
Fix some other graphical issues caused by the port from Xbox to PC.
Address issues with Jackal and stationary shields. Because of the rendering engine for Halo on PC, these issues canít truly be fixed. Jackal shields are now colored based on rank, and both those and stationary shields have a damage shader (I originally used CMTís Jackal Shields, but, switched to the Lumoria shields. Thanks to TM mapping team for those. Also, thanks to SOI7 for the damage shader for the stationary shields).
Fix the off-center elevation markers inside the Sniper Rifle scope, and modify the Sniper Rifle scope overlay to look more like the Xbox version (namely, saying 10x, as opposed to 8x, for the 2nd level of zoom).
Fix the issue with the shuttle bay door on A50 missing its off shader.
NOTE: There never was an off shader. Xbox used a shader_transparent_generic for the shield and used
the position of the door as a 3 second long function to slowly fade the shader to fully transparent.
Look here to see what I mean: http://i1151.photobucket.com/albums/o630/MosesofEgypt/shield%20shader_zpsvavaf3aq.png~original
Edited by MosesofEgypt on Jun 26, 2017 at 01:38 PM
So it's been bothering me that we've never had a port of the campaign to Halo Custom Edition that wasn't broken in some significant way that we could fix(missing multipurpose maps, broken cutscenes, broken scripts, no biped shadows, etc). I took some time today to fix this. As of this moment they are all uploaded to halomaps and are awaiting dennis's approval, but Michelle also uploaded them to her google drive too, so here:
Due to how the ui needs these maps to be named to load them(a10, b30, etc) that is still how they are named. Dennis, I know you have a thing against duplicate map names in the downloads, but it can't be helped here. I've at least prefixed the zip files with "Mo's Refined", so maybe use that for the download name and keep the actual XX0.map files named how they are.
Here are a few things to note about my conversion:
- All tags were ripped from the Halo PC campaign using Refinery, and I am quite certain
it rips perfectly(I played the entire campaign through on Normal to test it).
- Pistol and Plasma Rifle have had their projectile error cone FIXED in accordance with
Ifafudafi's information in this post:
- ALL missing multipurpose maps have been restored.
- Most diffuse and multipurpose bitmaps for the cyborg, vehicles, and weapons have been
replaced with 32bit versions extracted from the tags that shipped with the HEK.
This means the textures are the highest quality you'll ever see them, though it does
bump each maps size up by 30MB-50MB(cause they are higher quality AND they aren't
stored in the bitmaps.map anymore).
- Some scenery objects use shader_environment tags, and Halo CE does not render them
properly in fog. Halo PC does, but they broke this when making Custom Edition.
This can be seen by looking at one of the large human barricades in a10 near the start.
It's in a dark and foggy room, and it's tinted almost completely white(the fog color).
Nothing I can do about this without making new, different shaders(and I'm NOT doing
that for everything).
- Biped shadows DO appear, and any other bugs that you might've encountered while
extracting with HEK+ are not present.
For reference, this is a list of every multipurpose bitmap from Halo PC that has solid black in its red, green, blue, and alpha channels(meaning it's empty):
levels\a10\bitmaps\obj-tech rack multi.bitmap
levels\a30\devices\torpedo_bridge\bitmaps\metal flat generic bridge_multi.bitmap
levels\a30\devices\torpedo_bridge\bitmaps\metal strips narrow bridge_multi.bitmap
levels\a50\devices\light_ceiling\bitmaps\light fixture metal multi.bitmap
powerups\assault rifle ammo\bitmaps\ar ammo multipurpose.bitmap
powerups\flamethrower ammo\bitmaps\flamethrower ammo multipurpose.bitmap
powerups\pistol ammo\bitmaps\p ammo multipurpose.bitmap
powerups\rocket launcher ammo\bitmaps\rocket launcher ammo multipurpose.bitmap
powerups\shotgun ammo\bitmaps\sg ammo multipurpose.bitmap
powerups\sniper rifle ammo\bitmaps\sr ammo multipurpose.bitmap
scenery\halo\bitmaps\halo inner ring multi.bitmap
scenery\halo\bitmaps\halo outer ring multi.bitmap
weapons\assault rifle\fp\bitmaps\assault rifle multipurpose.bitmap
weapons\assault rifle\fp\bitmaps\display multipurpose.bitmap
weapons\flag\fp\bitmaps\flag fp multipurpose.bitmap
I figured out which ones were bad by extracting all the tags, and running a small script to check the tags. The script scanned for bitmaps with "multi" in their name, and checked if the pixel data was all black. For restoring these, I extracted the uncompressed multipurpose maps from the tags that shipped with the HEK in a similar way to how TeXource works. For the others, I extracted them from xbox maps with Refinery and used my Halo Bitmap Optimizer to convert them to PC. After converting to PC, I kept them as 32bit so more compression artifacts aren't introduced from decompressing dxt1/3/5, swapping channels, and recompressing.
NOTE: Because the flag is not in ANY of the campaign maps, its fixed multi isn't in any of these maps.
So yeah, essentially this is as good a port of the campaign as we can get for Halo CE, with enhanced fidelity on the most important textures to help make up for some of the things we can't make work.
Here's some pics to show you things that have had their multi's restored, textures that have been replaced with uncompressed versions, the shader_environment bug I'm talking about, and some Johnson butt grabbing.
Yes, I realize the screenshot was pasted wrong. I'm too lazy to fix it.
Edited by MosesofEgypt on Jun 27, 2017 at 04:54 PM
Honestly R93, I think he might be learning. He's posted twice on this thread and both are actually of reasonable length lol.
Sorry, it's my bad. Something's wrong with the portable install, so I just now pushed an update that comments out the buttons that allow you do so it. For now, don't select the install location and dont check "Portable install". The portable install might work for some people(worked for me) but it's not for some others, and I'm not sure why.
Likely months. Gonna try helping my friend with a game he's working called fantasy crescendo. I'm not a touhou fan, but he needs some help. Also, that bug i believe is caused by my definition not having the proper enum sizes for certain fields in the glow tag. Tweaking stuff now, though my pc just crashed lol. Should be ready for you to run the mek updater to get the fix in like, an hour. I originally saw that and in my tired stupor thought "eh, i think i remember it looking stupidly big like that" lol. So what are you thinking about the extraction overall though? I think Refinery beats HEK+ at this point lol.
EDIT: Actually, looks like my definition is correct as far as sizes go. Only thing that wasn't was the value for the NONE enum setting, which should have been -1 instead of 0. I'm checking halo pc to see if the glow is that stupidly big on there too, cause I don't see anything wrong with either of the sword's glow tags.
EDIT2: Okay, yeah, wow. So i'm guessing that glow is one of their oldest tags, cause the attachment enumerators aren't just byteswapped when they're converted to meta, they subtract 1 from them. The enum values in the tag are the same as you'd see throughout the rest of the library wherever you see function_input:
NONE=0, a_out=1, b_out=2, c_out=3, d_out=4
When converted to meta though, the glow tag uses these values:
NONE=-1, a_out=0, b_out=1, c_out=2, d_out=3
I tested this by looking at what the energy swords values got converted to when I extracted it. All values were -1, but when it got converted to meta in the recompiled CE map they were all set to -2, which isn't even a valid setting. This basically means I just need to add 1 to each of the values when I extract the meta.
EDIT3: Just tested ingame, and that fixed the bug. Turns out those 2 energy sword glow tags are the only ones in the entire campaign, and not every map has the energy sword in it. In any case, download refinery from the mek to get the fix. Version is bumped to 0.9.1.
Edited by MosesofEgypt on Jun 10, 2017 at 06:03 PM
Just finished playing through a10, a30, a50, b30, and b40 on normal. No bugs to report. I sent all 909MB of perfectly(as far as I can tell) ripped original campaign tags to killa so he can upload them for anyone who wants them. Someone should take the tags, fix the covenant shield door shaders(they look like garbage on pc) and restore all the missing multipurpose bitmaps and recompile them into better pc ports of the maps. We can rip the original bitmaps from all the tags that shipped with the hek using TeXource, so we can make all the weapons look as good as possible. I won't be doing anything halo related for quite a while, so consider this sorta a parting gift for the time being. Refinery will remain at v0.9.0 until I'm sure there are no ripping bugs(which you guys will need to notify me of). Cheers!
Edited by MosesofEgypt on Jun 10, 2017 at 04:42 AM
Everyone should re-download the mek and run the updater because i just fixed all the issues i talked about above. As far as i can tell, there are only some bugs ripping OS tags. When those are fixed, refinery will hit version 1.0 and be considered capable of perfectly ripping tags. As of right now i believe it can perfectly rip non-OS tags.
P.S. Someone leave a comment so i can post more updates.
Edited by MosesofEgypt on Jun 8, 2017 at 03:15 PM
I actually got into programming tools for that exact reason il duce. The bitmap converter was to properly convert all bitmaps to xbox format. I can eventually write a proper gbxmodel to model converter.
I've been thinking about making a program to basically do everything arsenic doesn't do and you just use arsenic for rebuilding the map. I've already decided that if I make the program, it'll be called Gallium. I chose this name because arsenic and gallium are highly poisonous substances, but combining them you get something extremely useful for electronics, gallium arsenide. I like naming stuff like this. Refinery is named because it extracts meta data and raw data from maps and carefully tweaks them to be working tags, just like a refinery carefully extracts precious minerals and materials from ore and other unprocessed resources and converts them into a usable form.
I remember someone here wanting to port the Halo 1.5 maps from xbox to custom edition, but couldnt because the lightmap vertices were compressed. Well good news everyone, I've incorporated decompressing those vertices into refinery!
Here's Crossways loaded in regular sapien.
Refinery can rip all the tags, my bitmap converter can turn all the bitmaps from xbox format into pc, and my model converter can turn all the models into gbxmodels. Cant do anything about the types of shaders and bitmaps that wont work on pc though. Well, the bitmap converter CAN fix the bitmaps, but you gotta specify them manaully. It still can't do anything about replacing shader_transparent_generic shaders since there is no one-to-one way to convert those to a usable shader.
Refinery is right here:
Edited by MosesofEgypt on Jun 7, 2017 at 03:46 PM
I've been working on fixing bugs with Refinery, and have gotten to the point that I can extract a10 from Halo PC and recompile it for Halo CE with only a handful of bugs. The current bugs are as follows:
* bipeds dont have shadows(doesnt seem to occur on all recompiled maps, but does on a10)
* keyes is about one foot too far to his left in one scene in the intro cutscene.
see picture below
* grenades dont chain detonate for some reason.
Flags in equipment and damage effect are properly set and the damage type is set
as grenade. Not sure what is going on.
* shaders on certain scenery are borked(not my fault, ce's rendering for
shader_environment is broken on scenery)
This is an overlay of a screenshot of the keyes scene I'm talking about. The Halo PC one is set to 50% transparency and overlaid on top of one from my recompiled a10.
Killa uploaded the recompiled map below so you guys can see what refinery can do right now. I can honestly say that in it's current v0.8.2 form it is most likely a more reliable and accurate tag extractor than the HEK+ at this point.
I literally just had to tell refinery to extract the tags to a folder, put tool in the folder, and run "tool build-cache-file levels\a10\a10". No tag editing required. Widgets, cutscenes, ai, scripts, everything works as it should(other than what i've mentioned above)
EDIT: If anyone would be so kind, I'd really appreciate a rip of the original campaign bsps from Halo PC using Sprinkle's real time editor; it wont run on my 32 bit machines. I should be able to fix the shadowless bipeds issue by comparing tags like giraffe said. Help me finish this guys, we're so close to having perfect tag extraction! When Refinery hits version 1.0 it'll mean I'm confident the extraction is 100% perfect(excluding super obscure bugs I would never think to test for).
Edited by MosesofEgypt on Jun 7, 2017 at 05:05 PM
I'll keep that in mind. I'm currently trying to debug this issue with the audio. At least I can take solace in the fact that Halo PC a10 raises an unknown exception when I try to compile it with tags ripped with hek+, but compiles without any errors when ripped with refinery.
Not planning to implement Halo 2, but if it wouldn't take too much work then it might be something worth investigating later.
Also, idk what the problem is with W7. I don't have these issue when I run it on my laptop with windows 7, so I can't really debug the problem. Try the console version of it since that runs with a console window open. If you have that same issue with any of the tools, change the extension from pyw(python windowless) to py. The extension forces it to run with or without a console window.
Finally, I'd like some help figuring out why there aren't shadows. Vehicles have them, but bipeds dont. Same goes for the rest of the bugs. If any of you can tell me what the issue is, It'd be really helpful.
Patronising? I want some actual feedback.
All right everyone, here it is. Refinery is NOT finished(hence the version 0.7), but it can do most of what you guys need it to do.
Here's a list of what Refinery can currently do:
* Load and extract tags from Halo 1 Xbox, PC, CE, PC Demo, and Yelo maps
* Load and extract tags from Stubbs the Zombie maps on PC and Xbox
* Load and extract tags from CE(not PC) resource maps(loc.map, bitmaps.map, and sounds.map)
* Display all pertinent information contained in the map's header, tag index,
scenario bsp headers, and Open sauce header(if it exists).
* Has 3 ways to view the map contents: hierarchy, class, and hybrid
Hierarchy sorts the tags by folder, like a regular directory structure.
Class sorts all tags by their class(bitmap, gbxmodel, weapon, etc)
Hybrid combines the two. It sorts everything by class, then inside
each class is a directory structure of all tags of that class.
* Extract all Halo 1 tags
* Extract all Open sauce tags
* Extract the cheape.map from Yelo maps
* Extract most Stubbs the Zombie tags(excludes vege, imef, terr, bipd, vehi,
and obje tag classes)
* Decompress and save Xbox maps
* Save any loaded non-resource map(rearranges structures and renames protected
tags so the map can be opened by other map tools)
* Allows you to rename tags and folders of tags in any loaded non-resource map(tags
have to at least partially exist in the map to rename them, i.e. they cannot be
in a resource map.)
* Allows previewing any tag in the map as if loaded in Mozzarilla.
Changes made to the tag will NOT be saved to the map. Exporting works in the
window, but rawdata will be in little-endian and other meta-specific details
will not have been tweaked, so don't expect to be able to properly import ANY
exported block into a tag.
* Allows extraction by tag class as well as by folder or individual tags.
Here's what is currently NOT implemented that I plan to add at some point in the future:
* Ripping from Yelo resource maps
* Fixing mangled class names(aka Zteam protection)
Most of the framework is in place to do this, I've just gotta fill in tons of offsets.
* Recursive extraction.
* Recursive renaming(you rename a biped/weapon/etc and it'll intelligently put most
tags it references in the same folder).
* Using hashcaches of collections of tags to detect and rename any matching ones in the map.
* Using heuristics to rename tags based on gleamed information(weapon/vehicle names in
the strings list, shaders/bitmaps/physics/collision/animations that go with them, etc).
* Using a config file to save your settings between sessions.
If you guys can think of anything else that you'd like to see(no bitmap/model/bsp preview please) then I'm open to suggestions.
Here are all the known bugs that I can think of right now(I plan to fix them):
* Tag explorers might not be sorted if you edit them(if you rename an entire folder
and it disappears from the tree, it will be added to the bottom instead of alphabetically)
So now that you've read all that, here's a few tips on using Refinery:
* After you've loaded a map you can extract everything to the "Default extraction folder"
by leaving the queue empty.
* You can double click an item in the map explorer(or select multiple and hit enter)
to either rename it/them, add it/them to the queue, or preview the tag(only if
selecting individual tags).
* Renaming a folder to one that already exists puts all tags in that folder into the new one.
* Renaming is case insensitive, and everything gets converted to lowercase.
It's just easier this way.
* Before you start adding things to the queue you should set the "Default extraction folder"
in the main window. This way when you add items to the queue, their extraction folder
defaults to it and you dont have to keep re-selecting it.
* If the project_yellow_globals tag doesnt extract, go into the settings and check the
"Use old project_yellow_globals definition" button.
* If you just want stuff from the bitmaps.map, sounds.map, or loc.map just load that map up
instead of an actual map.
* If you only want one type of tag(all bitmaps) then change the explorer to "class" view,
select the tag class(bitm in this case), and add it to the queue.
* You can edit the details of something in the queue by double clicking it.
Have some pretty pictures:
Let me know of any bugs you guys come across. I will try my darndest to fix them, especially if they are extraction bugs.
DOWNLOAD(Requires Python 3.4 or higher)
Click "Download repository" and extract the contents somewhere. Read the readme.txt for installation instructions. Refinery comes with the MEK. Whenever I have an update for it(which wont be very often at all) you'll likely need to re-download the MEK and paste the Refinery.py and refinery folders from the download over the old ones.
Edited by MosesofEgypt on Jun 20, 2017 at 03:01 PM
Yeah, everything open sauce is supported. There's even support for the old project_yellow_globals where you could reference a project_yellow_globals_cv tag. Everything is almost ready for you guys to test. There will be bugs (including some i know about atm), so dont expect everything to be perfect.
I'm currently finishing up the gui and adding special features, like renaming tags and directories before you extract anything. This way you can easily renamed protected tags and directories. I'm thinking about adding the ability to save the deprotected map, but i'll need to work out the logistics of it.
Well good news is that everything you guys have mentioned doesnt seem to be a problem with my extractor. I havent tested recorded animations or the biped shadows, but i tested everything else you guys mentioned and i see no problems. I've also got everything completely extracting. I just need to make a usable extraction queue widget and fix a bug with 4byte colors and then you guys can try it out.
Could you name the issues you've had with the hek+? It'd really help me if i knew what to look out for. I have no doubt there will be bugs for a while, so any that you know of with hek+ will be ones i should specifically try to avoid.
The tool is called Refinery, and it's a map extractor/deprotector for Halo xbox/pc/ce/demo and Stubbs the Zombie on xbox/pc. Basically every type of Halo 1 map(including open sauce yelo maps). The ui isn't totally done yet, and some of its more specialized functionality wont be implemented for a while(deprotection using hashcaches and fixing tag classes), but I feel like I'm far along enough with it that a beta release should be ready it in a few weeks or so. And yes EmmanuelCD, it will REQUIRE python.
There are a few bugs I have to fix in the extraction that I know of, but otherwise I can extract every tag(seemingly) just fine.
I've got the framework set up for fixing mangled tag classes, but it'll require a decent amount of time and careful checking to make sure I type in all the offsets correctly. Once this is ready for everyone to try I'd REALLY, REALLY like to get feedback on it. If something seems wrong, I need to know. It will be a while before you guys should trust it to perfectly extract stuff. Also, it wont be able to extract from yelo resource caches cause I dont know how those are used. I might add it later if I can, but for now the open sauce extraction will be limited to regular .map and .yelo maps.
It will be in the MEK repository for download when it's ready. You'll need to run the mek_installer and tell it to upgrade as well, since I'm making changes to the underlying libraries. Cheers!
Edited by MosesofEgypt on May 23, 2017 at 09:12 PM
Edited by MosesofEgypt on May 23, 2017 at 10:08 PM
Edited by MosesofEgypt on May 26, 2017 at 01:21 PM
Just added a tool to convert an entire tagset to/from 60/30 fps(save for a few exceptions). The tool is named Halo_Tagset_Fps_Changer. Check the readme for information.
Seriously, check the readme. I will point anyone to this post who complains about something that's explained in the readme, even if they majorly screwed something up or dont understand what it did. It's that important. This tool can and will edit a vast majority of the tags in any folder you point it to and change lots of values all over the place. I haven't been able to fully test it, so for a few months you guys should use it with caution.
Also, you WILL need to update the mek in order to use this.
Edited by MosesofEgypt on May 5, 2017 at 01:25 AM