XFileSharing Pro - Please Add a Raid 1 Option to the script.

What Do you think about this ??

Poll ended at Apr 24, 2011 9:56 am

Awesome !!! We Should have it.
5
83%
NAAAAA !!! Don't want it.
1
17%
 
Total votes: 6

Message
Author
micromaster
Posts: 19
Joined: Nov 08, 2009 9:55 pm

Please Add a Raid 1 Option to the script.

#1 Postby micromaster » Mar 10, 2011 9:56 am

It Would be perfect if you can add such a small option like this.
The point is Having one file stored on 2 servers.
Why would it be good?:
1- Fair load balancing (Hardware and traffic)
2- Fail Over support (If One Goes Down the second one Covers)
3- Regional mapping (One server could be in Europe and the other one would be in the US to get closer and faster to your client).

How it would look?:
The uploaded file would go to an upload server and from the there to be cloned to two download servers.

UPLOAD REQUEST ---->Upload server 1 -----> Download server 1A&1B
FILE LINK INFORMATION GOES BACK TO MAIN SERVER DATABASE.

The Download request page would give the downloader an option to download from one of two regions.

ankurs
Posts: 1054
Joined: Mar 10, 2009 2:34 am

Re: Please Add a Raid 1 Option to the script. !!!!!!!!!!!!!!

#2 Postby ankurs » Mar 10, 2011 11:36 am

micromaster wrote:It Would be perfect if you can add such a small option like this.
The point is Having one file stored on 2 servers.
Why would it be good?:
1- Fair load balancing (Hardware and traffic)
2- Fail Over support (If One Goes Down the second one Covers)
Will buy such a mod instantly.
micromaster wrote: 3- Regional mapping (One server could be in Europe and the other one would be in the US to get closer and faster to your client).
can be used, but not a must have; atleast for me
micromaster wrote: How it would look?:
The uploaded file would go to an upload server and from the there to be cloned to two download servers.

UPLOAD REQUEST ---->Upload server 1 -----> Download server 1A&1B
FILE LINK INFORMATION GOES BACK TO MAIN SERVER DATABASE.
UPLOAD REQUEST ---->Upload server 1A -----> Copes to server 1B
UPLOAD REQUEST ---->Upload server 1B -----> Copes to server 1A
... so on..

micromaster wrote: The Download request page would give the downloader an option to download from one of two regions.
keep it simple, just automatically select the closest server for the user using ip2location db or a server which has less network usage

chrda
Posts: 296
Joined: Sep 14, 2009 7:16 pm

#3 Postby chrda » Mar 10, 2011 1:47 pm

First all the post title doesnt make any sense.

Loadbalance of website (not files) is a easy process. And its not a script function. This can be both Software (Haproxy etc) or Hardware (Barracuda etc)

Loadbalance of files is a more complex area.
Loadbalance out from same datacenter
Loadbalance out from same location (.us but diff datacenter)
Loadbalance out from diff countries. (.us vs .eu)

There is so many variants to think about, and each site have different needs.

The costs will rise if you want a file mirrored on each server

The optimal way to do this is to use same datacenter, and use internal network for moving files, and external for pushing files.

Question is that if you want to just loadbalance the most popular files, or all files.

Using external network to move files is a bad idea, for most of the XFS users are getting the cheapest servers and overload them before getting new ones. I said most.. :)

My programmer have created a system for loadbalancing files, mainly used on a video site. This site uses Internal network for moving the files.
Without internal network this would be problematic.

So my point is here, different solutions for different niches. Not a simple mod buy Sibsoft. Creating this mod will probaly create a shitload of problems that would overwelm them.

As many of XFS users use sibsoft for admin services that they really should have inhouse. And a loadbalance setup is far from problem free.

Yea!

micromaster
Posts: 19
Joined: Nov 08, 2009 9:55 pm

#4 Postby micromaster » Mar 10, 2011 8:48 pm

Will thats why i said an option, so could be used when needed, in the meen time, my main consirn is providing the best service, that means having always a backup plan, having 2 server runing virtual raid would be costy i know, but in the same time would provide fault tollerence and give a bandwidth of 2 servers for single selected files.
For the datacenters issue, what if i dont want all my servers in one datacenter, what if it goes down, what if the whole countrys backbone goes down?
Down say it wouldnt, casue it could. "EGYPT - LIBYA - TUNASIA) over 10 days each, and libya is still till now in some places.
By this system im explaining alot of file hosting companys already use it.
Example : Hotfile.com !!
Yes they do, they have upload servers and download servers.
All in the same data center ? Nop

chrda
Posts: 296
Joined: Sep 14, 2009 7:16 pm

#5 Postby chrda » Mar 11, 2011 9:18 am

for the mirror part you dont need any mod.

<Solution>
2 identical servers.. rsync both ways every minute.
Either same datacenter, same area, or mirror a .us and a .eu server.

Put roundrobin OR geo location dns on both apache ip and nginx ip.

you need to disable old html upload and only use flash upload. (old html upload will work with geo dns system, but not with roundrobin)

This way you can use upload on both, and download on both.
</Solution>

If you had a server / network administrator that knows his job, he could do the mirror part you want in short time.

And if you want to compare hotfile, you need cashflow, your own script, your own programmers and your own server/network administrator(s)