XFileSharing Pro - Distribution of files on servers

Message
Author
speakerbox
Posts: 73
Joined: Aug 20, 2010 4:39 am

Distribution of files on servers

#1 Postby speakerbox » Oct 29, 2010 8:21 pm

Hello,

So here’s the problem I am having at the moment. I am getting users that are visiting my website and uploading six files at a time. The problem this is causing is that all six files are added to the same file servers because they were uploaded at the same time. Shortly after people start accessing these files and the connectivity for the particular server peeks while my other file servers are running at much lower speeds.

Is there anyway that these files can be distributed amongst all the servers even when several files are being uploaded at once?


Thanks!

chrda
Posts: 296
Joined: Sep 14, 2009 7:16 pm

#2 Postby chrda » Oct 29, 2010 10:02 pm

use gbit port and fixed amount of BW might be an option

100Mbit ports isnt very good these days :)

ankurs
Posts: 1054
Joined: Mar 10, 2009 2:34 am

Re: Distribution of files on servers

#3 Postby ankurs » Oct 30, 2010 4:59 am

speakerbox wrote: Is there anyway that these files can be distributed amongst all the servers even when several files are being uploaded at once?
would like to know the same, is it possible by code modification ?

mrperl
Posts: 65
Joined: Mar 06, 2010 11:40 am

Re: Distribution of files on servers

#4 Postby mrperl » Oct 30, 2010 8:55 am

My understanding is that different upload sessions load balance across different fs servers, but not files within a session.

One solution is to set max files of 3 per upload session.

Another is to make your site busier so that it will all average out. :)

speakerbox
Posts: 73
Joined: Aug 20, 2010 4:39 am

#5 Postby speakerbox » Oct 30, 2010 6:18 pm

chrda, I'm already on 1gbps ports.

I don't see how making my website busier will change anything. The problem is that multiple files can be uploaded to the same server in one session or more for that matter. Until a new file is added to the server the script won't switch the server that the files are uploaded to. This is making 4 or 5 files being added to a server at once and while other don't receive any. It doesn't take many connections to max out a 1gbps port. There must be a way to distribute files along 5 servers rather then having multiple files uploaded to the same server.

speakerbox
Posts: 73
Joined: Aug 20, 2010 4:39 am

#6 Postby speakerbox » Nov 02, 2010 7:03 am

Are there any options here??

admin
Site Admin
Posts: 1839
Joined: Mar 22, 2006 12:32 pm

#7 Postby admin » Nov 02, 2010 7:51 am

Unfortunately there's no way to distribute files from one singe upload session between different file servers as it's single stream.

But your thoughts about "4 or 5 files being added to a server at once and while other don't receive any". Why other servers will not receive any?
Next upload session will upload other "4 or 5" files on "other" servers.

Not sure where's problem here.

ankurs
Posts: 1054
Joined: Mar 10, 2009 2:34 am

#8 Postby ankurs » Nov 02, 2010 8:33 am

When we have option to upload multiple/bulk files, all will go to 1 server.

Isn't it possible to send the files to different servers ? with some code modification.

admin
Site Admin
Posts: 1839
Joined: Mar 22, 2006 12:32 pm

#9 Postby admin » Nov 02, 2010 10:35 am

As I said, that's not possible to send files on different file servers during 1 upload session. That's one stream of data which can go to only one server. We can't break it into parts.

Again, I do not see any problem here, because next user will upload "bulk of files" on another server. So, servers will be balanced.

ankurs
Posts: 1054
Joined: Mar 10, 2009 2:34 am

#10 Postby ankurs » Nov 02, 2010 10:43 am

admin wrote:As I said, that's not possible to send files on different file servers during 1 upload session. That's one stream of data which can go to only one server. We can't break it into parts.

Again, I do not see any problem here, because next user will upload "bulk of files" on another server. So, servers will be balanced.
ok thanks :)

speakerbox
Posts: 73
Joined: Aug 20, 2010 4:39 am

#11 Postby speakerbox » Nov 05, 2010 7:05 pm

The thing is that not every user will upload 4 or 5 files every session. So one server could receive five files at one while the other server only receives on. I’m having users who are opening two browsers and adding 8 files at once. Because they add the files at the same time they are all added to the same server. If the next user doesn't add the same amount of files then the file servers will be in-balanced. This is creating bottleneck of some of my file servers.

I know that there’s one website that distributes files evenly across all their filesharing servers regardless of how many files you add at once. I’m not sure what they’ve done to add this type of functionality but I can provide the url if need be.