XFileSharing Pro - Is it time for me to upgrade to a second file server? [i/o]

[venture89]

Hello All, We have been operating for about a month on a free video host site with nginx mod. However, recently I have seen HDD status going to critical on our file server during peak hours.

I was wondering if there is anything I can do before upgrading the server because as you can see from the attached image our up-link is only 50% used. I would also like to know if I should go with RAID 0 or RAID10 for the next File server (Performance wise which one is better?)

Also I think memory 8GB is fine but having more than that helps cache the files? How do I force nginx to cache size? is it automatically done


Servers specs: E3-1230/ RAM 8GB/ 3x3TB on a RAID 0(Data loss is not a huge concern for us at the moment)



Atop output:

[nomad]

RAID 10: because it would help alot with data as well as disk IO.
min 4 drives: get min 4 drives with RAID 10, so you will get 2x2tb of usable space and 2x2tb of backup if you go with 2tb drives. It would help a lot if you go with 8 drives but its up to you.
Memory: it help a bit to have more RAM depending on your configuration, specially for videos to be found faster (spotted by the server to be played)

This is what I would suggest.

[venture89]

Thanks ofr the suggestions. I was under impressions that write performance of RAID0 is better than RAID10. Also if going with RAID 10 HW Raid controller is a must?

Thanks again

[nomad]

Well RAID0 might be a bit better performance wise, but with RAID10 you dont have to worry about loosing data.

If you go with RAID0, if one of your hard drives fail your entire system will be gone because the RAID table will be damaged and unreadable (this can be manually repaired but its not easy).

With raid10 you can loose upto 2 drives without loosing any of your files (you can easily replace those drives, hosts usually do it for free).

Hardware and software have different advantages/disadvantages but the main reason people go with software is because its free and its easier to debug. Hardware RAID is NOT easy to debug.

[venture89]

Well RAID0 might be a bit better performance wise, but with RAID10 you dont have to worry about loosing data.

If you go with RAID0, if one of your hard drives fail your entire system will be gone because the RAID table will be damaged and unreadable (this can be manually repaired but its not easy).

With raid10 you can loose upto 2 drives without loosing any of your files (you can easily replace those drives, hosts usually do it for free).

Hardware and software have different advantages/disadvantages but the main reason people go with software is because its free and its easier to debug. Hardware RAID is NOT easy to debug.

If I tell you data loss is not an issue for us (have multiple file servers so there is no downtime and we do not offer any data security). Would you recommend RAID0 or 10?

Also how does strip size effect the i/o performance? any recommended values? I just want to know that I am getting the max possible use out of existing server before adding a new server which costs lot of money.

#atop attached indicate any areas for improvement?

[hostlife]

Well RAID0 might be a bit better performance wise, but with RAID10 you dont have to worry about loosing data.

If you go with RAID0, if one of your hard drives fail your entire system will be gone because the RAID table will be damaged and unreadable (this can be manually repaired but its not easy).

With raid10 you can loose upto 2 drives without loosing any of your files (you can easily replace those drives, hosts usually do it for free).

Hardware and software have different advantages/disadvantages but the main reason people go with software is because its free and its easier to debug. Hardware RAID is NOT easy to debug.

If I tell you data loss is not an issue for us (have multiple file servers so there is no downtime and we do not offer any data security). Would you recommend RAID0 or 10?

Also how does strip size effect the i/o performance? any recommended values? I just want to know that I am getting the max possible use out of existing server before adding a new server which costs lot of money.

#atop attached indicate any areas for improvement?

Still Raid 5 or Raid 10.

Say you have 2 servers with Raid 0. Even if 1 drive will fail on any server that server will go down (unless you will rebuild raid), also all data on that server will be lost. Say if you have 4x2TB data on it & you lost that server, your clients will leave you immediately. They will never trust on you even you don't offer data security

[venture89]

So are you guys suggesting that this is the max usage out of this server (given raid 0 performance is the best and our i/o status is critical in this server)?

[nomad]

Ok, well no you havent maxed out the current system but I would not suggest you to do so. There is a good chance that one of the drives will fail and you will loose all your data.

You cant be so black and white with your users, you cant just tell them you lost all their data! your rep will be ruined.
even if data security/loss of data isnt a factor here, I would still suggest RAID10. I am telling you all these info out of experience. I had to try all the solutions one by one and the final one that was satisfying was RAID10.

Regards

[PowerChaos]

well , if you look at hd performance i would suggest raid 10 (hardware )

here is more info about raid
http://en.wikipedia.org/wiki/RAID

also this can help you
http://www.icc-usa.com/raid-calculator.php

but if you go raid , then go raid 10 or if you do not care about raid
then use 2 hd's and split up the files over the 2 hd's (kinda raid0 only with the files splitted over 2 subdomains for load balance)

anyway
raid will always improve I/O as you use 2 disks to get the data instead only 1 disk

Greetings From PowerChaos

[venture89]

Thanks PowerChaos, would having two subdomains on a one server count as two instances of nginx installs? i am worried about cost aspect of doing that.

This would reduce the raid0 risk as well.