Possible Horizon database issue

[Nidhoggr]

I know that database issues do happen on MMORPG servers. But I believe this one may cause hardware damage and/or failures. It has happened only twice, but don't instances were while running Horizons. Symptoms include periods of time (usually only a few seconds) where nothing will response to the users' commands. These are random occurences not when trying to access the Vault or your inventory. This can happen when you are just walking (or flying).

This is the Error Viewer log. Notice multiple errors listed as: ntfs and disk. These are mostly write failures.


Details on the errors. While they don't crash my system because Horizons is not on main Windows hard drive, they might destabilize the system of others.

While you may think its a hard drive hardware failure on my part, I have tested my hardware. First I did a quick scan using Western Digital's Lifeguard tool. No problems. Next since my hard drive (D:/) is a Raid1 array I deleted the array so that I could perform a SMART scan of the drives. The reports stated my drive passed and 99% performance and 100% fitness. These are enterprise quality drive. I have 2 Western Digital 500GB RE2 SATA drives.

I suggest that everyone search their error viewer for ntfs or other filesystem type errors. To do this. Click the Start Menu, click the Control Panel option. In the control panel find the Adminstrator tools icon and double-click it. Now you see the Event logger icon, click that. The important information is under the System division.

[Steelclaw]

That is definitely something that isn't caused by HZ, but rather, HZ may trigger something else that's broken to act up.

You said you ran "quick" checks. Those just ask the drive "Hey, do you think you saw anything wrong?". You need to do a full surface scan. This can be done using the drive manufacturer's utility, just make sure you do the non-destructive test. There's a read-only version (the one you want) and a read-write one (which will destroy the data on the drive).

A few other places to look:

- bad memory: Grab a copy of memtest86+ (memtest.org) and run two full passes. If you get *any* errors, you have some hardware issues.

- overclocking: Are you overclocking? Overclocking your ram or PCI/PCI-E bus can cause problems like the above.

- overheating: How's the temp of your ram and hard disks? Cooked ram gets errors, overheated hard disks shut down and don't respond to read/write commands. If you touch either and go "OW!", it's probably too hot.

- marginal power supply: HZ uses your graphics card (which probably runs on 12v), so does your cpu, and hard disks. If your power supply is failing or slightly undersized, the extra load will cause trouble, like having a disk drive spin down unexpectedly, or random system crashes.


Note: *DO NOT* run chkdsk / "check disk for errors" / defrag until you have undone any overclocking, have tested your memory and have ruled out any bad hardware. Doing any sort of filesystem reconstruction with flaky hardware can result in corrupting a perfectly good filesystem.

[Nidhoggr]

Its not my hardware.
1) I have tested my memory with memtest86+ no problems, multiple passes.
2) I have not overclocked my system.
3) Not overheating. Using everest my my temperatures are fine. 44C (when running hz) for CPU, 42C GPU. 35C Motherboard. My raptor (Windows O/S drive) is running at 31C, so its doubtful my other hard drive is running hotter than that.
4) My PSU, is a PC&C 750w with 60A on +12v, definitely not the PSU. Everest reports: 3.3V (on +3.3V rail), 4.95V (on +5V), 12.16V (on +12V)

Chkdsk reports no problem with the partition. Chkntfs reports no problems with the filesystem.

I really doubt its my hardware since it only happened twice both instances while horizons was running. Last instance was 4/27/08. And horizons isn't nearly the most demanding piece of software running on my computer.

I did a full scan using lifeguard about 1 month ago, its not the drive. I really don't want to run the 2 hour test when I have no problems with it 99.9% of the time.

[Steelclaw]

You didn't mention when you took the hard disk temp measurements and power supply measurements. Those need to be taken after the system has been under load for awhile and given plenty of time to get nice and hot, preferably right when it starts to have momentary freezing. Hard disks get hotter when they're actively seeking.

How's your northbridge temp? (see above)

Power supplies behave differently under high load and high temp as compared to when they're lightly loaded and the fan is barely spinning. One factor, called ripple, increases with load and temperature. This is a very minute fluctuation in voltage at a frequency too high for your motherboard monitor to detect. You need an oscilloscope to measure it. Too much ripple and electronics that expect flat DC start to act a little weird. This is what separates the $100 750W power supplies from the $200 ones.

Fortunately, this is probably not something to worry about. I'm assuming you have a PC Power and Cooling power supply, which falls into the more expensive and better built category.

So, back to the easier and more likely things to check.

Check *both* hard disks. feel the casing of the drive and the area right below the spindle motor. Yes, one can be fine and the other one burning hot. It all depends on the temperature of the air immediately around the drive, how air flows around it, and sometimes even between drive revisions.

You said you tested your drive a month ago, yet the problem seems to have cropped up after you tested. Just because it tested fine yesterday doesn't mean it's fine today. Furthermore, some errors are correctable, which will result in a 'clean' surface scan after the offending sector has been marked bad and remapped to a spare. The process of remapping can leave the drive unresponsive for a few seconds or longer -- which will cause the hangs and error log messages you see.

Get a look at the S.M.A.R.T data and look at the remapped sector and error count on both drives. If the WD utility does not provide this information, it can be retrieved with a 3rd party utility. Whichever one is associated with harddisk1 is the one Windows is complaining about and the one you should expect to see remap or error counts on.

Also, if the drive shuts down from overheating or bad power, it might not log a S.M.A.R.T error. You'll have to listen for the drive powering back up next time it happens.

There are plenty of tools at your disposal to determine the cause of this problem. I've given you some suggestions and don't mind going over them in more detail.

The whole system freezing and Windows complaining about a very specific hardware component points to a hardware issue. The alternative, which you suggest, is that HZ is somehow capable of bypassing the security and abstraction layers put in place that prevent a user application from directly accessing hardware. On top of that, you're saying that HZ is somehow able to bypass hardware-level abstractions and cause physical hardware damage.

That is a very big and dangerous assertion to make. By saying that HZ can cause hardware damage, you will give reason for less knowledgeable people to not play or stop playing. You'll also waste VI's time dealing with those accusations and the time of people frantically checking to see if they're affected. Making those kinds of claims can lead to more trouble than you want when they turn out to be false and the people you upset want some compensation.

The very least you can do is take the 2 or 4 or even 24 hours to rule out the more common and explainable causes for your disk errors. Making the kinds of assumptions you mentioned in your last two posts isn't just lazy. In this case, it's irresponsible.

[Sigi]

That's a lot of handy information Steelclaw!

I was always under the impression that software never could cause hardware damage (real physical). Am I right?

[Steelclaw]

That's a lot of handy information Steelclaw!

I was always under the impression that software never could cause hardware damage (real physical). Am I right?

It depends on the hardware being controlled and what kind of damage you're thinking of.

All pata/sata drives have a hardware level security lock. If this gets set and the host system doesn't provide the proper unlock code, the drive won't let the host access it -- even after a power cycle. If you don't know what the code is because you didn't set it, the disk is now useless.

It's possible in some cases to reprogram the firmware of devices, like the motherboard, hard disk, or optical drive. This usually winds up bricking them, though I suppose they could be programmed to do bad things.

If bricking isn't good enough for you, there are some more destructive things to do. Most modern motherboards provide an API for messing with the cpu frequency as well as cpu and ram voltage. With the right privileges, those values can be cranked to 11 and overheat if not flat out burn up the ram or cpu. Even more dramatic: it's possible to mess with the video timings enough that a CRT monitor trying to display the video signal will burn out and possibly catch fire. (note: LCDs and some CRTs won't burn up, they will just not display the signal.)

There's also proprietary stuff, the damage you can do with that depends on how it was('nt) engineered.

Anyway, one of the above examples should be able to fit your definition of 'physical damage'. Do note, however, that in order to do these things today, there are all sorts of hoops to go through -- you have to start out with the intent of doing it, not accidentally flip a bit or two and boom, your hard disk has a bad sector.

[Sigi]

Thank you

[gopher65]

My computer has the same "HZ freezes for a few seconds during random actions" symptom, but there is nothing in the event viewer that I can find. So maybe the two things are unconnected? I haven't bothered to do memtest or checkdisks yet, but no part of my system is overclocked, and according to speedfan (if that gives an accurate reading of the temperature sensors) my computer is a long way from overheating.

I have HZ on a second drive and use Vista Home Premium. One interesting thing: I came across this same problem long ago when I installed HZ on one of my parents computers while I was visiting them during the Christmas break. That computer had a different OS (XP premium instead of Vista), different graphics card manufacture (ATI instead of nVidia), and it had HZ on its main harddrive, not the secondary one (I don't know about the motherboard or ram type or anything). The only similar thing that I can think of off the top of my head is that both systems had AMD processors. Dunno if that could cause something like this though?

[Hasai]

I was always under the impression that software never could cause hardware damage (real physical). Am I right?

I always wanted to know where that assumption came from. Certainly not from me.

It would be much more accurate to state that software cannot damage properly designed hardware; gear with built-in safeties that code can't touch.

Otherwise . . . well, I've seen 3330-series hard drives rip themselves right out of their floor mountings, magtape reels go flying across the room while spewing tape in all directions, RAM so hot you could fry an egg atop it, and a CPU literally explode.

So what do manufacturers do? "Hey! Let's set things up so the lusers can diddle with the clocks and voltages! That way, we'll sell LOTS more hardware!"

[Nidhoggr]

I not trying to blame Vi without cause. BTW: All reading from Everest were done through my g15 keyboard applet that I monitor frequently.

Have no idea whether the motherboard reading is the northbridge or an ambient motherboard temp. It doesn't specify.

Update on memtest86+ (13 complete tests / overnight); no errors

The warning could be due to the raid controller also, not copeing with the temp. lack of communication with the database, which I believe is the cause of the lag spikes.

And steelclaw I think I deserve the right to make a hypothesis. I am not blaming anyone now, just doing research. Many factors apply. Am I irresponsible to think logically here. Did I yell at the first symptoms... no. I checked my system after the first instance then waited for a second occurence before acknowledging the problem.

Speedfan has ALL reading on the hard drives listed at very good status.

Again I am NOT blaming Vi, just trying to get to the bottom of my problem. Maybe its my cache mode and maybe its the raid controller.

edit: Right now NO DAMAGE has been done, it is just a 'hiccup' in communication.

[T-Sar-Goth]

Just a quick note. Not to plug any particular software package but there is some software that I've used that can non-destructively low-level format a drive to detect and in some cases repair bad sectors on a hard drive. The software is called Spinrite. You may find more information about it here http://www.grc.com/spinrite.htm

[Terao]

I got problems on my system (before it blew up). System freezed, I got error messages in my log files,

• I've tested my harddrives, they where fine
• I tested with memtest and everything seemed to be okay
• Speedfan temps seemed to be okay (don't believe what speedfan says!!!)
• two day later the symptoms got worser, blue screens, system hang ups and reboots - and only when I played HZ. You may need to know that HZ is the only program I use at home that does a real havy system load.
• I tested again with memtest and it showed that one of the ram got damaged, I sticked in two new ones everything seems to be working again
• another three days later it got even worser, heavy system hang ups, blue screens, rebootes like never before, I switched everything off
• after one more day the system won't even boot, something on the motherboard got burned
• I bought a new motherboard (I never buy Asus again!) and a new cpu, same cooling as before everything is running fine now, no error messages or any other unusual issue

If you see it that way, yes HZ burned out my system - but it was the motherboard where the cooling relevant parts where simply too small, and that's a fault of the designer of this motherboard and not HZ fault, take a benchmark tool or something that produces real heavy CPU load and check if you get the same errors as you get while playing HZ.

[aine]

I have noticed similar occurrences myself. Walking along (pressing the forward key) and I just stop moving, the game kind of freezes, (can't tell if the entire pc does, as I seldom have any other program open when IS is on), etc. Haven't run any checks, just assumed it was lag or whatever (I am not very technically minded, can you tell? Heh). Only posting so you know it happens to another, as well...

[GalemThawn]

Looks like a possible problem paging memory from RAM to Disk. Insure your 2nd harddrive, where your page file is located, is not having any problems. Also check that you have the appropriate speed of RAM for your motherboard.

[Nidhoggr]

Checked the disks (D:/) and the RAM is compatible (DDR2-800). Motherboard supports up to DDR2-1066.

[Steelclaw]

And steelclaw I think I deserve the right to make a hypothesis. I am not blaming anyone now, just doing research. Many factors apply. Am I irresponsible to think logically here. Did I yell at the first symptoms... no. I checked my system after the first instance then waited for a second occurence before acknowledging the problem.

You're more than welcome to state an observation as you are to make a hypothesis, however, a hypothesis isn't a conclusion, and you shouldn't draw conclusions based on an untested (or under-tested) hypothesis. That's what got me annoyed -- you made a conclusion (with very serious implications if it were true) without demonstrating sufficient testing. When you refused to do any further testing and just assumed your hardware 'must be good', that got me upset.

Anyway, you have cleared up your position and so long as you're willing to work on finding the cause, rather than finding proof that HZ is the problem that caused the errors in your log, I have no problems and I'm willing to help.

(Note that gathering facts and drawing a conclusion based on them is different from having a conclusion and trying to find facts that support it.)

That being said, can you post screenshots of the contents of the S.M.A.R.T tab from speedfan for each of the disks associated with harddisk1? (If it's a raid array, I'd like to see the S.M.A.R.T data for both disks in the array.)

[Nidhoggr]

You really need to lighten up, I NEVER stated that Horizons was to blame. I stated it was a possibility. If its seems like I had already made my case, I apologize to you and the devs. I gave you my opinon (not a full blown conclusion), it shouldn't be a personal vendetta against the developers or yourself.

Give you a few minutes to delete the array and post those results.

edit:
First off this screenshots are a tad on the big side. Stupid mspaint to blame.



First drive listed in the last screenshot is the raptor, the second and third are the WD RE2 500GB

For an alterative source here in the WD lifeguard utlity report

[Steelclaw]

Thanks for posting that.

The reallocated sector count on both drives is 0, which means the drive didn't take time off to reallocate a sector. Given that that value is 0 and that none of the other error counters are non-zero, it's probably safe to say that neither drive had a surface error and you can probably rule those out.

Assuming you managed 13 passes of memtest with no errors, the cpu and ram are probably safe to rule out too.

That leaves your motherboard (and the controllers on it), power, cabling, and maybe the raid drivers.

Unless you can think of ways to conclusively eliminate those, you'll have to do more observation when the freezing happens again.

[Ahael]

Could be a intermittent power related problem, screwing up signalling while the system is under heavy load. I've seen quite a few computers / motherboards / PSUs with faulty capacitors. And sometimes there's no visual indication about them being faulty.

[Balthizar]

Thanks for posting that.

The reallocated sector count on both drives is 0, which means the drive didn't take time off to reallocate a sector. Given that that value is 0 and that none of the other error counters are non-zero, it's probably safe to say that neither drive had a surface error and you can probably rule those out.

Assuming you managed 13 passes of memtest with no errors, the cpu and ram are probably safe to rule out too.

That leaves your motherboard (and the controllers on it), power, cabling, and maybe the raid drivers.

Unless you can think of ways to conclusively eliminate those, you'll have to do more observation when the freezing happens again.

The cabling will be the easy one to check first and you can check the power supply if you already have a spare you already know is good to go.

If thoughs are both good send your mainboard back and ask for a new one. The company that makes your mainboard probly will not send you the Tech book and scematics I asked one time and they laught at me