extracting textures from the World cache

[Cobal]

Back on tazoon.com there was a thread with map making resources.
In this thread was a link to a program a HZ player had made that extracted a certain section of textures based on coordinates entered.

The filename was texmapgui.exe. I still have it but it is corrupted.

Does anyone still have the Link or the executable?

[Fridlekh]

http://mysite.verizon.net/fridlekh/H.../TexMapGui.exe

Summary:
This program stitches together LOD terrain textures into a single bitmap file, based on coordinates provided by the user.

Usage:
Run the exe. Put values into the fields shown. Press the "Do Stuff!" button. (Blame Shavnir.)

Limitations and bugs:
This program expects a LOD texture size of 128. If you have played with this value and have LOD textures of different sizes in yoru world cache, your map may be corrupt or the program may crash.

I don't have Tab Ordering in place yet. You have to use your mouse for now.

I don't have typechecking in place yet. Putting letters instead of numbers in the number fields will probably crash the program.

I don't have folder verification implemented yet. Feeding the program an invalid HZ directory or server name will result in a black map.


Release notes:
2/11/2004 - First release
There's a few unimplemented features. Those blank spaces in the form contain dissabled controls.




has it really been 9 months since I made this thing? Did anyone want features that never made it in?

[Vanaondo]

Hello There.

Im Pekka that makes the map package. I tested your program but I only get black maps for Unity. I know you wrote that we get black maps if we write wrong directory or wrong server name but maybe you can point out what I do wrong?

In Sweden, default install directory is Program and there is where I have Horizons.

Hz dir: C:/Program/Artifact Entertainment/Horizons/
Servername: unity
Start x,y: 31000 21000 stop x,y: 36000 26000
File Name: c:\test\ncon.bmp

So what might I have done wrong?

[Tabea]

I pm`ed Tree on that too ...

Is there a possibility to define which set of textures to draw?
for example only the forests to get theyr rough destinations
Pekkas map pack has dark green areas where forests are, but snowy areas like mahagra/kirasanct, deserts like tazoon and even blighted areas are problematic.
The destination is needed regarding a new map I try to do.

[Fridlekh]

dumb Q ... did you change the dir in the box from it's default /program files/ to /program/ ? Also, could you copy and paste the full path to your world cache directory? perhaps it's different that what I expect...

[Fridlekh]

I pm`ed Tree on that too ...

Is there a possibility to define which set of textures to draw?
for example only the forests to get theyr rough destinations
Pekkas map pack has dark green areas where forests are, but snowy areas like mahagra/kirasanct, deserts like tazoon and even blighted areas are problematic.
The destination is needed regarding a new map I try to do.

forrests? no. nor can I do resource regions. Those are not defined by textures or masks, or even sector defs, but are sent to the client by the server dynamically. with the mask files, it's possible to only render certain types of textures, roads, grass, lava, dirt, etc ... but I never built that funtionality into the original mapping program... I'm too lazy to go to the trouble of indexing all the mask numbers in relation to the textures they represent.

[Starcuss]

I too got black screens when trying to extract the textures and after running Filemon I noticed why too. It wants a ChunkTexture_xxxxx_xxxxx.aei, where the x's represents the coords and on my computer I don't have any of those stored.
So my question is how do I get so Horizons store those files? I remember an old post at Tazoon.com and playing "offline" does that have something to do with it?

[akb]

Seems obvious, but it's easy to forget or have it incomplete ...

To be able to extract data from the world_cache, you first need to fill the world_cache with data. To do maps for only a small part of the world, it's a medium amount of work, but to get the whole worldmap, you need to cover an area of about 20x28 squarekilometers (for the main continent including thenew North Continent and the new two western isles) plus some more for the two separate areas (one of 8x6 squarekilometers for the four starter islands and one other for the starter area of new chars)which takes up a total of around 2.3 GB in the world_cache.

There is a new option in Horizons to clear the world_cache (or part of it) but when i tried that option, nothing seemed to be cleared and after restarting Horizons, the flag always was cleared again. But watch out to not accidentally erase your world_cache in the future if you still want to use it later.

Besides the size of the cache, it is also a very huge number or files. To give an estimate: almost 12k files in 1700+ directories, and another44k+ texture files in one other directory. All this data and the huge directories of course slow down the system and every recall/gate.

some more info on filling the cache for those who want to do it themselves:
(btw: I'm Daj and did texture maps of the North Continent and the western islands)

Since there is lots of watery area on the map and also many obstacles (mountains, cliffs, monsters, etc)when trying to cover the world by foot, this "cache filling" should be done by an adult dragon who can "fly the world in stripes". When I did that while using maximum view distance and flying at sealevel, i could see (get the textures) for a distance of about one kilometer to each side. btw: one texture file has 128x128 pixels and covers an area of only 100x100 meters, and thus you must have that small area in view range to get it transfered to the cache.

But the stripes to fly have to be smaller than 2 kilometers (width of 20 textures)for two reasons: most of the time you can't fly exactly straight north/south or east/west on a specific exact coordinatebecause of inaccuracies when setting the course, but most of all, the view distance is calculated 3D and thus the horizontal distance shrinks when flying higher, eg around 700 meters (7 textures on each side)only when flying 700 meters above ground (and some mountains are much higher). Thus you should fly stripes at most every 1500 meters (and fill some missing areas later) or even fly 20 stripes of 1 kilometer width only.

When flying at max speed (91 if you don't have wings with tech V) and with max view range, you will do only a little more than 400 meters per minute, for a total of up to 70 minutes for one stripe of 28 kilometers. Some stripes are shorter (the map is not exactly rectangular), but getting to the next stripe takes some time too. Thus all this mapping will easily take around 24 hours of pure flying time until you got those 2.3 GB of landscape data (btw: most of the time, I got an almost constant dataflow of 55 kilobits on my DSL line; no task for modem or ISDN users). If you carefully select stripes andsomehow jam the "climb"-key, you might even be able to fly AFK in intervals of up toan hour :-)

Do you still want more info on how I did maps with all this data? Read on:

To get an overview whether some textures are missing, and to have complete maps of the world, I used TexMapGUI to do maps of 8x8 kilometers which (as a BMP file) take up circa 320 MB each, then scaled them down to 800x800 pixels and saved them as JPG with some graphics program to have an overview and easily read the coordinates of missing parts (black holes) in the textures. Later I did another set of maps with 2x2 kilometers each, which fit the size of the original maps in the maps/istaria folder, and which therefore can simply be copied to that folder without any additional work, replacing the old maps there (eg for the western islands which are "in" the old maps), or with little additional work (adding a few coordinates for additional map segments to the north of the istaria map for the north continent).

When doing those maps, you have to enter the coordinates, and here it gets a little tricky: TexMapGUI rounds coordinates up and down to the nearest 100 because each texturemap stands for an area of 100x100 meters, and the coordinates given are not the left/bottom and right/top edges of the created map, but they are the bottom/left coordinates of the texturemap segments. example: to create on overviewmap of the new two western islands, you would use the coordinates16500/22500 for bottom/left and19400/25400 for upper/right (instead of 19500/25500) to do a map of exactly 3x3 kilometers.

Since I wanted to also include the edges of the map (to see them and notfall off the edges which forces a recall), I added 100 meters in each direction: coordinates from16400/22400 to 19500/25500 which results in 32x32 textures (30x30 for 3x3 km, plus one extra along each edge) or a picture size of 4096x4096 pixels. Here is a smaller version of this picture, scaled down to 25% of the original size (with the center 960x960 for the 3x3 km area): TexMap-WestIslands.jpg

[Starcuss]

Excerpt from the log when using Filemon:

185112:47:10TexMapGui.exe:1652OPEND:\Horizons\worl d_cache\Unityhorizons\terrain\lodtextures\ChunkTex ture_25900_34000.aeiPATH NOT FOUNDOptions: Open Access: All
185212:47:10TexMapGui.exe:1652OPEND:\Horizons\worl d_cache\Unityhorizons\terrain\lodtextures\ChunkTex ture_25900_34000.aeiPATH NOT FOUNDOptions: Open Access: All
185312:47:10TexMapGui.exe:1652OPEND:\Horizons\worl d_cache\Unityhorizons\terrain\lodtextures\ChunkTex ture_26000_34000.aeiPATH NOT FOUNDOptions: Open Access: All
185412:47:10TexMapGui.exe:1652OPEND:\Horizons\worl d_cache\Unityhorizons\terrain\lodtextures\ChunkTex ture_26000_34000.aeiPATH NOT FOUNDOptions: Open Access: All
185512:47:10TexMapGui.exe:1652OPEND:\Horizons\worl d_cache\Unityhorizons\terrain\lodtextures\ChunkTex ture_26100_34000.aeiPATH NOT FOUNDOptions: Open Access: All
185612:47:10TexMapGui.exe:1652OPEND:\Horizons\worl d_cache\Unityhorizons\terrain\lodtextures\ChunkTex ture_26100_34000.aeiPATH NOT FOUNDOptions: Open Access: All

As you can see TexMapGui is looking for a folder called lodtextures and then the ChunkTexture_xxxxx_xxxxx.aeifile, however on my computer I don't have the folder lodtextures or the ChunkTexture-files, thus the maps will turn up black.

[akb]

D:\Horizons\world_cache\Unityhorizons\terrain\lodt extures\ChunkTexture_26000_34000.aei

That would be the exact directory which on my computer was filled with more than 44000 files of size 49 kb each. Since (in theTexMapGUI boxes) you have modified the path to your horizons folder to point to D:/Horizons instead of C:/Program Files/etc/etc/etc/Horizons, and also set the server (else it would not look in Unityhorizons subdirectory) I have to assume that you did that correctly and on purpose :-) and since it is windows, unity should give the same results as Unity, and also the "/" vs "\" seems to be no problem on this program.
And the coordinates are for the southern edge of Trandalar, right next to the Ice Island, which should be correct too if you tried to do a map of that area :-)

This leaves only two possible problems which i can see:
Either you have installed Horizons twice and/or Horizons stores data on another partition, and thus you don't get that directory created, or you did the mapmaking after closing Horizons and the new flag did its work on your computer ("There is a new option in Horizons to clear the world_cache (or part of it) but when i tried that option, nothing seemed to be cleared and after restarting Horizons, the flag always was cleared again. But watch out to not accidentally erase your world_cache in the future if you still want to use it later.")

When did you run TexMapGUI? While HZ was running or after you logged off?
If your ram permits, you might try running TexMapGUI while still logged in, and also check the flag to delete (part of) the world_cache in the ingame options.
You might try to start over with the cache too, by deleting the whole tree "unityhorizons" (or simply renaming it to any other name), and while you are at it, check every letter again on the path (partition D, maindirectory Horizons in the rootdirectory)

And (as general hint, not for you, since your directory didn't exist at all)just to be sure that the area really was seen and corresponding textures exist,look at your current coordinates as shown in the compass or with /printpos and use those coordinates plus and minus 500 or 1000 for a first test.

btw: when first using the program i got strange results. i had used the left boxes to enter X-from and X-to, and the right for Y-from and Y-to, which of course couldn't work since the left boxes are for X-from and Y-from (bottom/left coordinates ofbottom/left texture) and the right boxes are for X-to and Y-to (bottom/leftcoordinates of top/right texture). Thus don't feel too secure and check for even the most stupid errors you would never do yourself:-)

[Fridlekh]

I too got black screens when trying to extract the textures and after running Filemon I noticed why too. It wants a ChunkTexture_xxxxx_xxxxx.aei, where the x's represents the coords and on my computer I don't have any of those stored.
So my question is how do I get so Horizons store those files? I remember an old post at Tazoon.com and playing "offline" does that have something to do with it?

*edit for exact terminology used in options*

Turn on Terrain LOD in the video options. If you think the LOD textures look ugly, set the switch distance to max, you'll still cache textures for areas you see.

As for why you don't notice anythign when you "clear cache" it's because it ONLY clears the sector .def files and NOT the cached textures or mask files. Clearing the sector defs can fix problems like CTDs related to corrupted def files.

[Starcuss]

Turn on Terrain LOD in the video options.

That solved the problem. Thanks [:D]

[Cobal]

[:D]

Thanks millions :)
I haddnt suspected an answer anymore, havent been in the map forum, for weeks.

*flies of to to get a new complete cache*

[Tree]

I would be very much intrested in the complete worldcache [:D]

[akb]

I would be very much intrested in the complete worldcache [img]/Web//emoticons/emotion-2.gif[/img]

I would be willing to send it to you to replace my exact texture maps with your nicer 3D maps, but the amount of data makes it almost impossible to send via internet :-(
The data is transfered compressed to the client since it takes around 20 hours to fly across the whole world and datastream is only around 56 kbit most of the time when approaching new sectors, but even at my full connection speed of DSL, downloading the full 2.3 GB of data would take 5+ hours and sending it would take 42 hours.
I hope you find someone living next to you who can help, or someone who has a server available with a very fast connection where you can download it.

[Fridlekh]

The world cache compresses pretty well since all teh the data is in raw uncomressed formats. Try doing a solid RAR archive with a large table. I compressed my 693 MB orderhorizons dir to 263 MB for example.

Still large, but only 37% the size. I suspect 7zip might do a better job...

[Tree]

This will probably end up as ~450 meg 7zipped