Excelsium

Possibilities: - It only affects users with this "bug" either on the sending or the recieving side - which side is undertermined. - If its not on the recieving side - The problem with recieving the files is not indefinite - related to which users are online sharing the files - a few of those users may have this problem sending the file. either way bandwidth and slot wastage is happening. - peers sending and recieving the same chunk(s) of files ad infinitum. the wastage potential will increase if the crash fix you release does not address this (no crash = chunk(s) continue their infinite transmission.) It would be irresponsible to let this wastage continue and soon perhaps increase by an order of magnitude. IIRC RevConnect did not crash at all or nowhere near as often, the waste occurs in that client which led me to testing and switching to StrongDC++ and then ApexDC++ as said. I will be testing segmenting again in the next release - If I encounter this issue again I will post a FRAPS/whatever video of the bandwidth+slots being wasted.

Im replying in the feature requests forum. Blegh I should have kept it here and started a new thread, oh well. I have asked for those threads to be moved here.

Yea it just seems extremly bizzare, unbelievable (For it to be a bug in hardware/os/etc) that it would happen repeatedly on the same files with the same TTH, but succeed on all other files (in Segmenting mode) I understand your saying its a bug somewhere in my machine (and my previous machines) but for it to be that specific seems unbelievable. My thinking is that bugs like that happen randomly on random files Its clear for reasons stated that this is an incredibly specific precise "bug" with no apparent randomness to it. Theories: (maybe posted some of these earlier). 1. StrongDC segmenting download technique*1 is detecting that peers copies or chunks are corrupt*2 when they clearly are not*3 (False positives) and is preventing the completion of that chunk and therefore the completion of the download. *1. Has and is using a corruption detect technique of its own *2. A small percentage of files, perhaps a tiny percentage of the overall files shared using DC++ *3. As proven by testing 1.2. Something else in StrongDCs technique is causing false positives in an extremly specific manner. 1.3. StrongDC is corrupting specific files, detects its own corruption and prevents completion. Theories of bugs in my machine: 1. An amazingly specific bug ??? 2. ?? add more later Reply to post in crash+bug forum. QUOTE(Zlobomir @ Mar 30 2007, 05:58 PM) * Again I have to double post, excuse me BM. But what about the file in question is really ok, runs perfectly, etc., after downloading in some other way. And Excelsium, please send me again some small files for tests, or even better, attach them here. I think Its a bit of situation A and a bit of something else. What I mean is. I think there is a problem with the technique causing false positives or real corruption due to itself (a bug somewhere in the technique of Strongdc) - And then it crashes after the false or real corruption that it has falsely detected or caused. See the list of thories above, this is my current one. On emulation: I don't use it for those hubs or any others - If I come accross a hub that requires it to connect, I don't use that hub because of some issues I heard of. Users downloading from when it crashed?: 1. The crashes are not very frequent since I usually spot the problem and pause the file before it crashes. 2. I have not been logging the users it was downloading from when it crashes. I'm probably not going to do any more testing with segmenting mode until a new version of strong/apex is released or someone has suggested configuration tweaks I have not tested. On my machine: I'm still thinking about the hard drive clusters and will add to my reply if needed. But for now heres what I know. 1. My hdd has no problems: defragmented, 0 bad sectors/problems reported by S.M.A.R.T, 0 problems reported by windows tools 2. How can that problem affect a specific set of files with same TTH indefinetly?, files are written in whatever free space (clusters) is available.. which is random. All the other details have already been posted I'm sure. for example "Perfect copies of files can be downloaded using normal mode but cannot be completely downloaded using segmenting mode for the same specific files with same TTH" Again 95+% succeed in segmenting mode, the exact same (No ramdomness*) 1-5% have the download not completing problem in Segmenting mode. *Randomness characterizes the data layout on a disk, and RAM aswell. But this is clearly a precise bug, maybe you have some details on how a hw or sw defect or perhaps configuration setting outside of StrongDC/apex could result in this very precise bug affecting a set of files with the same TTH indefinetly? Another reason why this workaround should be implemented: (At the very least the client should pause the download and warn the user to prevent slot and bandwidth wastage, and save the user time identifying the corrupt files) When the crash is fixed the client will be left to continue to download the same chunk(s) over and over again until the user returns and spots it, of course sometimes they wont spot it. - Wasting slots and bandwidth in the network. For me thats 18hrs a day of unsupervised waste.. except weekends. Of course I'll leave segmenting disabled but what about others who dont know about these issues yet? I know I've been experiencing it for a long time with various machines but only just taken action to deal with it - If I had not taken action I would waste many more gigabytes of bandwidth without the crash fix but with the crash fix I would waste 2x or 3x+ more. I've a 10mbit connection, I know I've easily wasted 100s of GB's of my own and others bandwidth because of my ignorance of the only known existing workaround (segmenting set to disabled), for example a good percentage of that on popular files with 100+ sources - always maxing my connection. I started using RevConnect in 2004 - I've been wasting slots and bandwidth for all that time, about 2 months ago I moved to StrongDC>Apex (easier on the eyes) because of these issues but was disappointed to find they are in these clients aswell, so I began my quest for a workaround here, luckily I found one not ideal.. but it works (Disabling segmenting). The potential that this kind of waste can happen has to be stopped and of course the ongoing waste by people who have the issue, are ignorant of it and have not disabled segmenting - via an automated method. Options: 1. Detect corruption and pause the file, put error in the error column. 2. Detect corruption and switch technique for the file.

Ok thats a misswording on my part. the facts below and above still stand. I'd like some detail on how it would be a HDD/memory bug that causes the exact files with same TTH to fail over and over in segmenting mode when starting from scratch even starting with a default clean install of apex/strong. Again - downloading these files in normal mode complete fine and are verified (checked and matched TTH) against a copy another source (bittorrent) and are fine in extensive subjective testing. So I think that destroys the theory that these files are corrupt and the segmenting technique is detecting corruption in the peers (all peers) copies and preventing completion.

I've never had a problem with normal downloading . And its always the same files with the same TTH that fail in segmenting mode even starting from scratch. deleting temp file etc. the other 95+% of files dont have this problem in segmenting mode. It just takes a few other people to try downloading the same TTH in segmenting mode to prove it either way.

I check the TTH for the copy downloaded (downloaded completly without any errors) from dc++ peers (with segmenting disabled). and the copy downloaded with Bittorrent. - they match I've viewed both copies in full of a few such cases, and scanned quickly through the rest - I've tested the TTH in all cases and all have the same TTH, but why would there be a problem? they all have the same TTH. I use PowerDVD 7 with the default install of codecs from divx.com to view the files, no other codec packages etc are installed. I am happy with ApexDC++/strongdc running without segmenting mode - I have no problems whatsoever, but it would be a nice extra to have workable segmenting. (reply to post in bug+crash forum)

The smallest files I download are very rarely some image files and audio files, and I don't remember having problems with these - If I did I probably just erased from the queue and forgot about it. Currently I only use DC++ for files 100-300mb video files, maybe the odd video file 10-100mb My earlier post: Update: Incase you havn't encountered some problem files for testing here are some examples: they are only a problem in segmenting mode. search " mushishi 03 .a" and get file with TTH: HBBAHIWV56PEJQD5H6S6J4WAFQ2J3ELRNYBMJUA magnet:?xt=urn:tree:tiger:HBBAHIWV56PEJQD5H6S6J4WAFQ2J3ELRNYBMJUA&xl=189263872&dn=%5BC1%5DMushishi_-_03%5BXviD%5D%5BF334DBDF%5D.avi search " ghost in the shell 24 .a" and get file with TTH: FY6T6IN2MEZCXWCNDL6HMAWP4HX6I2RRXRNBY2Y magnet:?xt=urn:tree:tiger:FY6T6IN2MEZCXWCNDL6HMAWP4HX6I2RRXRNBY2Y&xl=186181632&dn=Ghost+in+the+Shell+-+Stand+Alone+Complex+-+24+%5BLMF%5D.avi hubs with these files: testhub.otaku-anime.net:555 public.otaku-anime.net:555 walhalla.otaku-anime.net:555 shinto.total-anime.net:666 (requires registration to search)

Thanks for helping my poll choices (I edited the poll so my votes were removed): x Yes, with "corruption detected at position" in the status bar, but I have tried and succeeded in downloading the same file(s) in regular DC++ client or with Muti-Source disabled. x ApexDC++ crashed. x I dl LOTS of files.

I've posted something in the feature request forum and a poll in this forum :P

I've finished testing "Enable multi source" set to "disabled". Good news: No corruption messages or crashing.. all downloads completed and are perfect as tested against Bittorrent releases - remember they all fail in segmenting mode. Problem solved I guess - I no longer have to use regular DC++ alongside strongdc or apex to catch its segmenting failings. I'm hoping that someone builds a Detect and switch feature as I described in full earlier so others and I could use segmenting mode - Or builds a segmenting technique that is 100% instead of 95-99% The DC++ segmenting problem has been around for years - I had the same issues in RevConnect (another segmenting DC++ client). But only recently bothered to do anything about it since the amount of files I transfer has been steadily rising. I have changed PCs, reformatted etc several times during that period.

Hi I want the "Disconnecting slow downloads" to behave as follows: Disconnect if speed from user is below 3kb/sec reguardless of total file speed And never remove anyone from queue. I've included a screenshot, have I done it correctly?. Thanks a lot. :P

I have tried "Manual settings of number of segments set to "1" and had the same problems. But I don't think I've tried "Enable multi source" set to "disabled". I'll come back with the results later. If this works then I'd like my "Detect and switch technique" idea to be added to Strong, if you really feel you cannot fix the corruption any other way.

Strong could be modified or features added so it dosn't corrupt files - fact.

These files people are sharing are clearly not corrupt or garbage data since I have watched them all the way through without any problems whatsoever (DivX,Xvid Avi files) and have checked the TTH against the bittorrent release where possible. Maybe your trying to touch another topic here - "Crap" is a matter of taste. I dont want censorship in my P2P client software thanks.

Thanks we got to the bottom of it: There are solutions - "Fix" StrongDC++'s download techniques. - Include the standard DC++ download techniques as an option. (I'd like this to be done anyway as a backup) Ideal solution as said above I think - if you cannot fix strongs techniques: Detect corrupted file when it says "corruption detected at position" then erase the file and temp file then Re add it and start again using standard DC++ download techniques Detecting the corrupt file would be easy - just see which one is downloading the same chunk over and over.

If your theory is correct then there needs to be a way of disabling it since Its having false positives on corruption detection or better yet, fix it. Either way I tested (downloaded) a set of known "problem" files in Strong and Apex - all fail I tested (downloaded) the exact same set of files in regular DC++ in the same hubs from the same users - all succeed and are perfect copies of the files. - I have even checked the TTHs against files that are available on bittorent and have downloaded with bittorent. So you think my perfect copy is "crappy" eh? what a joke. And this testing continues My Conclusions so far: any file that fails to download in strongdc also fails in apex but suceeds in regular dc++ (and is a perfect copy). Thank goodness for the regular dc++ client - I'll download whatever files I want thankyouverymuch.

Downloads > Queue > Segment Downloading > "Enable multi-source" set to "Disabled" There you go :)

DC++ regular client is working (sucessfully completes all downloads) with these problem files and ALL other files without corruption messages so I have to disagree, I have zero problems with other segmenting download: http, bittorrent or anything else at all on this PC.: So can you elaborate in full detail please on why you are so certain it "must be caused by some failure in your PC"? Obviously your claim cannot be proven or disproven until there is Lots of results from lots of testing, you havn't yourself yet, correct? To those affected: My workaround for this is to run plain DC++ alongside strong/apex, delete the problem files from strong/apex queue and add them to your dc++ - temporarily disconnecting from hubs in strong/apex and connecting to hubs in dc++ until the strong/apex problem files complete sucessfully and then disconnect from hubs in dc++, connect to hubs in strong/apex

1. Once "corruption is detected at positon" the download always fails from all sources, even with files with over 150 sources (users). 2. I download about 50 files a day all of those are in the 100mb-300mb range, 3-5% fail as said above.

Great. Single segment downloads are giving the same corruption messages - no problem in regular DC++.

I'm still not having any problems with the regular dc++ client, is there a way to force selected files to have only one segment, will you add this feature if its not there and that is if this bug is not completly fixed. although maybe its not a bug in segmenting but a bug elsewhere in the download process, but we wont know until we test with single segment downloading. Otherwise I'm forever stuck using regular dc++ client because of those 3-5% of files that fail in apex, strong & revconnect. Also the client should tell you exactly what file the corruption is detected rather than the "corruption detected at position" and having to observe the transfers and find the one thats downloading the same chunk repeatedly then at least I could continue using the client by queuing those quickly identified corrupted files in regular dc++, it would take way too much time otherwise. Update: Settings>Downloads>Queue>Manual number of segments "1" works to set new downloads to single segment I'll report back results later. Idea: If the client can detect which file is corrupting then it could for example pause the file to prevent crashing or automatically erase and re add the file in single segment mode.

@Big Muscle: As far as I know It happens only when downloading those few problematic files, It occurs at random intervals whilst these problematic files are running, downloading the same chunk of the file over and over again, crashing after downloading the same chunk maybe 10 times or 100 times. I have tried these same files in all segmenting clients I know of - always fails. Tried these files in plain vanilla DC++ with no segmenting and it worked first time, no corruption messages, no crashes. Small possibility remains that there is a few problematic users on the hub and downloading from them with segmenting or regular client hoses the download and causes crash and that they were all offline when I tested with regular client and thats why they all worked, doubt it though. Update: Incase you havn't encountered some problem files for testing here are some examples: search " mushishi 03 .a" and get file with TTH: HBBAHIWV56PEJQD5H6S6J4WAFQ2J3ELRNYBMJUA magnet:?xt=urn:tree:tiger:HBBAHIWV56PEJQD5H6S6J4WAFQ2J3ELRNYBMJUA&xl=189263872&dn=%5BC1%5DMushishi_-_03%5BXviD%5D%5BF334DBDF%5D.avi search " ghost in the shell 24 .a" and get file with TTH: FY6T6IN2MEZCXWCNDL6HMAWP4HX6I2RRXRNBY2Y magnet:?xt=urn:tree:tiger:FY6T6IN2MEZCXWCNDL6HMAWP4HX6I2RRXRNBY2Y&xl=186181632&dn=Ghost+in+the+Shell+-+Stand+Alone+Complex+-+24+%5BLMF%5D.avi hubs with these files: testhub.otaku-anime.net:555 public.otaku-anime.net:555 walhalla.otaku-anime.net:555 shinto.total-anime.net:666 (requires registration to search)

I have this exact same problem - is there anyway to download these problem files? someone has made a patch or workaround? or? I have attached my crash report and included a bit of it here; Always the same error recurring on same few TTHs. Code: c0000005 (Access violation) Version: 0.4.0 (2006-12-24) Major: 5 Minor: 1 Build: 2600 SP: 2 Type: 1 Time: 2007-03-19 04:42:30 TTH: YBP45KNMY2PE64J2ZH7A77UJFMDBI4A4FN657VI d:\cvs\apexdc++\apexdc\client\filechunksinfo.cpp(100): FileChunksInfo::addChunkPos d:\cvs\apexdc++\apexdc\client\chunkoutputstream.h(64): ChunkOutputStream<1>::write d:\cvs\apexdc++\apexdc\client\filteredfile.h(137): FilteredOutputStream<UnZFilter=0x02961018,1>::write d:\cvs\apexdc++\apexdc\client\downloadmanager.cpp(692): DownloadManager::on d:\cvs\apexdc++\apexdc\client\userconnection.h(411): UserConnection::on d:\cvs\apexdc++\apexdc\client\speaker.h(68): Speaker<BufferedSocketListener>::fire<BufferedSocketListener::X<3>=0x03FAFE00,unsigned char *=0x03FAFEA8,int> d:\cvs\apexdc++\apexdc\client\bufferedsocket.cpp(260): BufferedSocket::threadRead d:\cvs\apexdc++\apexdc\client\bufferedsocket.cpp(528): BufferedSocket::run d:\cvs\apexdc++\apexdc\client\thread.h(133): Thread::starter f:\rtm\vctools\crt_bld\self_x86\crt\src\threadex.c(348): _callthreadstartex f:\rtm\vctools\crt_bld\self_x86\crt\src\threadex.c(326): _threadstartex exceptioninfo.txt