XFileSharing Pro - Taking long to generate links after upload?

Message
Author
speakerbox
Posts: 73
Joined: Aug 20, 2010 4:39 am

Taking long to generate links after upload?

#1 Postby speakerbox » Nov 02, 2013 5:47 am

Hello,

At the moment we are having some problem when links are being generated after uploads are completed. On occasion the it will take 8 to ten seconds to generate links when the upload finishes.

We have several quad-core servers with 8GB of ram and 4x2TB HW raid 10. All the servers are around 30 percent capacity and the network is only 20 percent used. We thought that the problem is with disk IO but I'm not sure as disk IO seems alright. It disks read speed seems to max out at 20mbps during peaks times.

Anyhow, we have tweaked nginx aswell as apache but we still have this same problem. It's very frustrating and we'd like to resolve it.

Can anyone recommend anything hat may help? Ankurs?

ankurs
Posts: 1054
Joined: Mar 10, 2009 2:34 am

#2 Postby ankurs » Nov 02, 2013 6:18 am

I am guessing this happens only with big files and not small tiny files.

Is your temp folder on same partition as uploads folder ? If not try that, because after upload files are moved from temp folder to uploads folder.

When your moving 1gb file from temp to upload folder, if its on same partition its just a move; but if its not then the file is written to uploads folder which is limited by your write speed.

Also make sure your raid is optimized.
http://www.computerweekly.com/RAID-chun ... erformance

speakerbox
Posts: 73
Joined: Aug 20, 2010 4:39 am

#3 Postby speakerbox » Nov 02, 2013 5:26 pm

Hello Ankurs,

The temp folder is on the same partition as the rest of the script under /.

What is a normal IO wait for a busy server?

Can you recommend some software to monitor our disks? We installed smartmontools how-ever our disks as SMART compatible.

Very frustrating problem...

speakerbox
Posts: 73
Joined: Aug 20, 2010 4:39 am

#4 Postby speakerbox » Nov 02, 2013 9:55 pm

Hello,

It's getting a bit more busy now so I thought I'd post atop and iotop stats.

Image

Image

When we took those screen shots we had around 215 concurrent connections. Any suggestions would be great.

ankurs
Posts: 1054
Joined: Mar 10, 2009 2:34 am

#5 Postby ankurs » Nov 04, 2013 9:08 am

97% disk usage, you need to upgrade to a better setup with more disks

ufkabakan
Posts: 332
Joined: Apr 13, 2011 9:37 pm

#6 Postby ufkabakan » Nov 04, 2013 10:53 am

Your Problem about Disk IO
And how you getting up 1500~ Disk Read IO with only 300~ connection?

I can see 1000~ connections can create 1000~ Disk Read IO on my FileServers. Something is wrong on your Server setups.

speakerbox
Posts: 73
Joined: Aug 20, 2010 4:39 am

#7 Postby speakerbox » Nov 04, 2013 11:42 pm

Perhaps nginx isn't configured properly?

In the past we've had less powerful boxes that had no problem with disk IO on raid 10.

We also have some raid 1 servers with two hard-drives and the performance is identical to that of the servers with four disk raid 10. That shouldn't be the case especially since the raid 1 servers are at 90 percent capacity.

Any suggestions on how to proceed?

Thanks