XFileSharing Pro - Anti-dupes system

Message
Author
adiga
Posts: 177
Joined: Jul 23, 2009 2:27 am

Anti-dupes system

#1 Postby adiga » Oct 14, 2009 3:38 pm

how work Anti-dupes system ?
this features on v1.5

I have a member uploading same file several times during the night

admin
Site Admin
Posts: 1839
Joined: Mar 22, 2006 12:32 pm

Re: Anti-dupes system

#2 Postby admin » Oct 15, 2009 5:50 am

You can see several similar files in file list, but physically its 1 file on HDD.

This "anti dupe system" calculating file hash for each file and if system already have such file, then it's just placing "link" to existing file. So, no duplicate files will be physically stored.

adiga
Posts: 177
Joined: Jul 23, 2009 2:27 am

#3 Postby adiga » Oct 15, 2009 11:59 am

Thank you very much admin

ufkabakan
Posts: 332
Joined: Apr 13, 2011 9:37 pm

#4 Postby ufkabakan » Feb 22, 2014 11:57 am

I have same MD5 files on 3-4 times on differnet FileServers Disks.

So, this feature working only one FileServer?

Ankrus told me a fix about it:
its not a bug, anti-dupe system is server specific only

you can remove the server specific condition by editing fs.cgi

Code: Select all

   my $ex = $db->SelectRow("SELECT * FROM Files WHERE file_size=? AND srv_id=? AND file_md5=? AND file_real_id=0 LIMIT 1",$size,$server->{srv_id},$md5)


to

Code: Select all

my $ex = $db->SelectRow("SELECT * FROM Files WHERE file_size=? AND file_md5=? AND file_real_id=0 LIMIT 1",$size,$md5)
This changes can be fix rely?

And its looks only for new Uploaded files, whats about already uploaded same size and same MD5 files?

ufkabakan
Posts: 332
Joined: Apr 13, 2011 9:37 pm

#5 Postby ufkabakan » Feb 24, 2014 11:04 pm

I tested Ankrus Fix but doesn't work :(

This code give same Real ID for all new same files (size + md5) but still looks files go to servers and when you try to download same file from different server give this error:

Code: Select all

Error happened when generating Download Link.
Please try again or Contact administrator.
(ERROR:no_file)

I need urgently a Fix for this issue:
Need 1* Checking new uploads for size + MD5, if same file is already here, dont create new one, use oldones Real id with new name etc.

Need 2* Checking All files for same size and same MD5. If a file different copies, let alone one orijinal and remove other copies from other servers and use only orijinals Real ID for others download link.

ankurs
Posts: 1054
Joined: Mar 10, 2009 2:34 am

#6 Postby ankurs » Feb 25, 2014 8:03 am

ufkabakan wrote:I tested Ankrus Fix but doesn't work :(
remove comment "#" from the below line

#$server->{srv_id} = $ex->{srv_id} if $ex;

ufkabakan
Posts: 332
Joined: Apr 13, 2011 9:37 pm

#7 Postby ufkabakan » Feb 25, 2014 1:34 pm

ankurs wrote:
ufkabakan wrote:I tested Ankrus Fix but doesn't work :(
remove comment "#" from the below line

#$server->{srv_id} = $ex->{srv_id} if $ex;
YES! :) Its working now. Thank you very much
However we need still 2nd problem. How check old files for same size and same MD5 :( Because i thing i have TB size of same files...

Example of 1 file have 4 copys on 4 another FileServers and 1 Cloned on same server:

http://www.ufkabakan.com/img/clonedfiles.png

ankurs
Posts: 1054
Joined: Mar 10, 2009 2:34 am

#8 Postby ankurs » Feb 25, 2014 6:18 pm

ufkabakan wrote: However we need still 2nd problem. How check old files for same size and same MD5 :( Because i thing i have TB size of same files...

Example of 1 file have 4 copys on 4 another FileServers and 1 Cloned on same server:

http://www.ufkabakan.com/img/clonedfiles.png
No easy fix for this, contact support for customization; they should be able to write a small script to do this.