Segmented downloading techniques

[Excelsium 0]

Sorry that I must say it. But are you really blockheaded or just you aren't able to read what I have wrote to you in all of your threads ???

so again,

!!!!!!!! ERROR OF HARDWARE HAVE NOTHING TO DO WITH SEGMENTED DOWNLOADING !!!!!!!!!!

Now you are able to understand it??????????

And why normal downloading is accepted if it allows to leave corrupted files your HDD and allows to spread these corrupted files over the network (reason of having two many "same" files with different TTHs in the network). Segmented downloading allows only leaving and sharing 100% correct files.

You are right only in one sentence - the corruption message should display also filename of corrupted file and maybe it will be changed to this way.

And only one bug needs to be fixed - crash in FileChunksInfo. BUT IT HAS NOTHING TO DO WITH THE CORRUPTION BECAUSE IT CRASHES REGARDLESS THE CORRUPTION (understand that it crashes sometimes when there's no corruption and just you are the case that it crashes when there's corruption)

NOW YOU UNDERSTAND IT???

Sorry for that but I really don't know what I should write to you, because if I writes anything, you will still say your rubbish as "there's bug in segmented downloading because I have corrupted files" etc

!!!!!!!! ERROR OF HARDWARE HAVE NOTHING TO DO WITH SEGMENTED DOWNLOADING !!!!!!!!!!

Correct but there are bugs in the way your segmenting download technique attempts to repair corrupted temporary files in a percentage of cases.

Crash bug:

Its possible I'm wrong on the crash bug - its the only bug that can be conceded - I've added a note to the opening post saying that the cause(s) of the crash is/are unproven:

Possibilities:

1. It crashes randomly regardless of the "redowloading the same chunk ad infinitum bug".

2. It crashes randomly and because of the "redowloading the same chunk ad infinitum bug".

NOW YOU UNDERSTAND IT???

Sorry for that but I really don't know what I should write to you, because if I writes anything, you will still say your rubbish as "there's bug in segmented downloading because I have corrupted files" etc

I have had a small amount of corrupted temporary files yes caused by error(s) in hardware yes with the remote possibility that the cause was software error(s) elsewhere on my machine and outside of DC++ Client.

But your segmenting technique should be able to handle corrupted temporary files as is the case in other p2p clients, clearly at least in some cases it does not - checkout the Buglist in the opening post - e.g. "redownload same chunk(s) ad infinitum etc"

You are right only in one sentence - the corruption message should display also filename of corrupted file and maybe it will be changed to this way.

This is nearly the entire point of this thread.

I see you have acknowledged the user should be informed of which temporary files are corrupted - This is a critical thing. - This prevents the waste of the users time and his own and others resources. :whistling:

Perhaps you have accepted the waste (see opening post) is not acceptable as I have?

I have added a shortened rewording of the thread for you at the beginning of the opening post. - Cut by more than half and removed the possible crash bug.

So sit back and see whether he fixes it. He does it for free remember, and so do we.

Point taken .

Added rewording for Big Muscle at the beginning of opening post - it is short too read - about 30% of the original

there can be no real excuses for it being too long now.

[Big Muscle 1]

Correct but there are bugs in the way your segmenting download technique attempts to repair corrupted temporary files in a percentage of cases.

Sorry, but redownloading of corrupted block is a bug??? Sorry, but we aren't such stupid to leave the corrupted file on your HDD so you could share it and spread this corrupted file over the network.

But your segmenting technique should be able to handle corrupted temporary files as is the case in other p2p clients, clearly at least in some cases it does not - checkout the Buglist in the opening post - e.g. "redownload same chunk(s) ad infinitum etc"

This is nearly the entire point of this thread.

every p2p client does the same if there's corrupted block, it will be redownloaded (in most of cases from different user, if there's any with free slot). It's only your problem that the redownloaded chunk will be again corrupted when it's saved to your HDD. We won't support this corruption and you must accept it, or you should go away from DC network, because we don't need the user who wants to spread corrupted files over the network.

[Excelsium 0]

Sorry, but redownloading of corrupted block is a bug??? Sorry, but we aren't such stupid to leave the corrupted file on your HDD so you could share it and spread this corrupted file over the network.

every p2p client does the same if there's corrupted block, it will be redownloaded (in most of cases from different user, if there's any with free slot). It's only your problem that the redownloaded chunk will be again corrupted when it's saved to your HDD. We won't support this corruption and you must accept it, or you should go away from DC network, because we don't need the user who wants to spread corrupted files over the network.

- I don't want to spread corrupted files over the network.

Sorry, but redownloading of corrupted block is a bug??? Sorry, but we aren't such stupid to leave the corrupted file on your HDD so you could share it and spread this corrupted file over the network.

every p2p client does the same if there's corrupted block, it will be redownloaded (in most of cases from different user, if there's any with free slot). It's only your problem that the redownloaded chunk will be again corrupted when it's saved to your HDD. We won't support this corruption and you must accept it, or you should go away from DC network, because we don't need the user who wants to spread corrupted files over the network.

Redownloading the same block ad infinitum is a bug.

1. Client should prevent this behaviour and inform the user which file is corrupted beyond repair (by the client)

Client = StrongDC++

[Big Muscle 1]

- I don't want to spread corrupted files over the network.

I thought it, but your posts show something different :whistling:

[Big Muscle 1]

Redownloading the same block ad infinitum is a bug.

1. Client should prevent this behaviour and inform the user which file is corrupted beyond repair (by the client)

Client = StrongDC++

yes, it's the bug, but in your PC. We won't finish the file until all chunks are 100% correct.

OK, let's say we stop redownloading after 10 tries. But what then? The only option is that you will delete the file from queue, because we don't let you to finish it with corruption.

I'm only interested in fixing the crash during the download. You would be helpful if you have helped us to find the bug in code. And not only spread lies about bugs in segmented downloading.

[Excelsium 0]

I thought it, but your posts show something different

1. I don't share files I download until verified by myself (TTH + Subjective tests).

2. Of course I will use a workaround to your bugs to prevent said waste.

yes, it's the bug, but in your PC. We won't finish the file until all chunks are 100% correct.

OK, let's say we stop redownloading after 10 tries. But what then? The only option is that you will delete the file from queue, because we don't let you to finish it with corruption.

I'm only interested in fixing the crash during the download. You would be helpful if you have helped us to find the bug in code. And not only spread lies about bugs in segmented downloading.

Denial that bugs exists - Of course others and I disagree.

The bugs are listed in the opening post and elsewhere.

I've given all my replies already before this post, its upto Big Muscle If he wants to coverup his bugs with endless denials.

It probably doesn't matter what I say to Big Muscle now I think Big Muscle will do anything he can to 'cover it up'.

Anyone new/returning to the thread.. just checkout my opening post, flames and XP2s threads in this forum.

Sorry for respost of my earlier reply but heres where Big Muscle admitted something needs to be done.

You are right only in one sentence - the corruption message should display also filename of corrupted file and maybe it will be changed to this way.

This is nearly the entire point of this thread.

I see you have acknowledged the user should be informed of which temporary files are corrupted - This is a critical thing. - This prevents the waste of the users time and his own and others resources. :whistling:

Perhaps you have accepted the waste (see opening post) is not acceptable as I have?

yes, it's the bug, but in your PC. We won't finish the file until all chunks are 100% correct.

OK, let's say we stop redownloading after 10 tries. But what then? The only option is that you will delete the file from queue, because we don't let you to finish it with corruption.

I'm only interested in fixing the crash during the download. You would be helpful if you have helped us to find the bug in code. And not only spread lies about bugs in segmented downloading.

I forgot to add a point to the reworded version I made.

One option instead of just pausing the download after a set number of chunk redownloads:

- If the corruption handling technique fails to repair a corrupt temporary file it should delete the download from the queue and re add it to the queue automatically. (Starts over with a new temporary file)

- The user could be given this option when the corruption is detected or set it to do this automatically.

[Seeker 0]

sorry. I won't read it.

a) it's too long

:whistling: I don't understand why you have created 4 same topics about the same problem

c) Strong/Apex doesn't cause the corruption so write to your HDD/memory/OS manufacturer to provide you the tools for fixing the bug

I also did not read the whole thread, its like a licence agreement, nobody reads those. anyway. i recently had a similar problem with uTorrent and downloading, everything i downloaded was corrupted, even though i recheched and redownlaoded. checked my memory it was fine, checked hdd, was working fine except something crewed it up, most likely defragmantation. i dont know if it was a filesystem error, just disk blocks rewriting each other or whatever, but OS didnt read the files properly. only solution extended format of the disk and reinstall of OS. conclusion applications that used to work right didnt just go berserk and stop working right, its probabyl the same "bug" i had.

[Excelsium 0]

I also did not read the whole thread, its like a licence agreement, nobody reads those. anyway. i recently had a similar problem with uTorrent and downloading, everything i downloaded was corrupted, even though i recheched and redownlaoded. checked my memory it was fine, checked hdd, was working fine except something crewed it up, most likely defragmantation. i dont know if it was a filesystem error, just disk blocks rewriting each other or whatever, but OS didnt read the files properly. only solution extended format of the disk and reinstall of OS. conclusion applications that used to work right didnt just go berserk and stop working right, its probabyl the same "bug" i had.

I agree - reading just the rewording at the of the opening post and recent replies is enough to get a jist of the bugs.

Also I have no visible problems or issues anywhere else in my system including other p2p applications such as eMule and Bittorrent (uTorrent) in this case.

- In this case only 1-5% of all files I download with the segmenting download technique have corrupted temporary files beyond repair of itself the client (StrongDC++)

[Seeker 0]

I agree - reading just the rewording at the of the opening post and recent replies is enough to get a jist of the bugs.

Also I have no visible problems or issues anywhere else in my system including other p2p applications such as eMule and Bittorrent (uTorrent) in this case.

so? for me shareaza worked fine while utorrent wasnt. the disk bug was still there, but didnt affect all applications.

Its clear that segmented downloading and/or normal downloading did not create your problem, nor will you insinuate to the developers to drop segmented downloading by default. your error is most likely in filesystem, maybe you should ask microsoft about the corruption error, i presume you have one of the windows's installed.

[Excelsium 0]

so? for me shareaza worked fine while utorrent wasnt. the disk bug was still there, but didnt affect all applications.

Its clear that segmented downloading and/or normal downloading did not create your problem, nor will you insinuate to the developers to drop segmented downloading by default. your error is most likely in filesystem, maybe you should ask microsoft about the corruption error, i presume you have one of the windows's installed.

1. I'm not trying to "insinuate to the developers to drop segmented downloading by default".

I am trying to get Big Muscle or someone else to fill in the gaps in the corruption handling techniques. (corruption handling techniques are incomplete resulting in said consequences)

I guess I'm happy for the bugs to be reffered to as "gaps in the corruption handling techniques"

2. again:

- In this case only 1-5% of all files I download with the segmenting download technique have corrupted temporary files beyond repair of itself the client (StrongDC++)

From what I can gather other users with this issue also experience it at 1-5%.

I'm suggesting techniques such as "Delete and re-add to queue"

In cases that the same chunk(s) have been redownloaded "too many times".

- Slight repost: At the very least I'm requesting that the corruption message be changed - this has been accepted by Big Muscle, he hasn't said whether he will do it or not though.

[Big Muscle 1]

So solution:

- When I find why it crashes, I will fix the crash.

- I will add a name (or at least TTH) of corrupted file to "corruption detected" message.

and that's all.

It will not erase and readd the file, because in 80% it won't help and users will not only complain that it deletes their files. These clever TTH leaves verification has been made to redownload only corrupted parts and not whole file (which will be corrupted with same probability as the first downloaded file).

Btw, if you read my first post to your topic, you will find that I said it can be caused by software error too (not only hardware). Like in the past, ZoneAlarm caused corrupted downloads which resulted in infinite "Rollback inconsistencies" messages (only with normal downloading, not for segmented downloading!) and noone complained! They just uninstalled ZoneAlarm and everything worked then.

[Big Muscle 1]

btw purpose of what I wrote and you didn't understand:

if wouldn't have a bug in your PC then you wouldn't need any change in corruption detection routine.

[Excelsium 0]

btw purpose of what I wrote and you didn't understand:

if wouldn't have a bug in your PC then you wouldn't need any change in corruption detection routine.

Law of error rates in the users hw/sw as stated in full already.

- We dont live a perfect world where these errors don't exist.

- Other users experience this

- The more users there are, the more users who will experience this to a significant % (1-5%)

- Therefore the more users who require the said gaps in said techniques filled.

I hope I dont have to repost more.

I have suggested techniques to fill these gaps

- You have already accepted one of them (Change the corruption message).

So solution:

- When I find why it crashes, I will fix the crash.

- I will add a name (or at least TTH) of corrupted file to "corruption detected" message.

and that's all.

It will not erase and readd the file, because in 80% it won't help and users will not only complain that it deletes their files. These clever TTH leaves verification has been made to redownload only corrupted parts and not whole file (which will be corrupted with same probability as the first downloaded file).

Btw, if you read my first post to your topic, you will find that I said it can be caused by software error too (not only hardware). Like in the past, ZoneAlarm caused corrupted downloads which resulted in infinite "Rollback inconsistencies" messages (only with normal downloading, not for segmented downloading!) and noone complained! They just uninstalled ZoneAlarm and everything worked then.

1. "because in 80% it won't help": There is absolutely no way you can be sure of that percentage. - 95-99.99999999% (estimate) of the users files download successfully.

2.

One option instead of just pausing the download after a set number of chunk redownloads:

- If the corruption handling technique fails to repair a corrupt temporary file it should delete the download from the queue and re add it to the queue automatically. (Starts over with a new temporary file)

- The user could be given this option when the corruption is detected or set it to do this automatically.

- It is optional - the automated way being disabled by default, user is presented with dialog box.

Example dialog box "The temporary file for your download "foo.text" has probably been corrupted beyond repair, Would you like to re download it?"

- Even if the redownloading chunk bug caused by corrupted temporary file beyond repair bug happens in for example 1 in 100000 of all downloads - this would be useful. :whistling:

3. Moved

4. I have acknowledged the law of software error rates already.

[Big Muscle 1]

1. "because in 80% it won't help": There is absolutely no way you can be sure of that percentage. - 95-99.99999999% (estimate) of the users files download successfully.

if you don't have arguments against, you are trying to quibble. Hmm. Better don't say anything.

I will write it in different words: THAT THING WON'T BE IMPLEMENTED BECAUSE IT WON'T HELP IN NEARLY ALL CASES. Now you understand it?

- Even if the redownloading chunk bug caused by corrupted temporary file beyond repair bug happens in for example 1 in 100000 of all downloads - this would be useful. :thumbsup:

Have you ever written a lot of source code??? If yes, then would know that something which would be useful in 1 of 100000 cases, is totally useless and it's no point in doing it.

Btw just a personal question: You doesn't have more useful things to do than being all day in this forum and trying to force everyone to think that there is some bug in segmented downloading?

Personal advice: If you, instead of being in this forum, try to find a cause of your corrupted files, then you won't need to write here anymore.

[Excelsium 0]

if you don't have arguments against, you are trying to quibble. Hmm. Better don't say anything.

I will write it in different words: THAT THING WON'T BE IMPLEMENTED BECAUSE IT WON'T HELP IN NEARLY ALL CASES. Now you understand it?

Have you ever written a lot of source code??? If yes, then would know that something which would be useful in 1 of 100000 cases, is totally useless and it's no point in doing it.

Btw just a personal question: You doesn't have more useful things to do than being all day in this forum and trying to force everyone to think that there is some bug in segmented downloading?

Personal advice: If you, instead of being in this forum, try to find a cause of your corrupted files, then you won't need to write here anymore.

- You've just said you'll hand out the key to do it (TTH in corruption message)

*it - do something automated to prevent said waste.

- I can of course do this myself, others can too.

- Hence solution is found (when/if you release this "feature") - it took awhile to get there.

- I don't respond to insults.

- I will say this though.. slight repeat:

I'm not trying to force anyone to think anything.

My only goals of this thread were to:

1. Prevent said waste caused by said gaps in said techniques.

- Achieved by using another download technique.

- I will use the current segmenting technique when the "TTH in corruption message" feature is released.

2. Build an automated process of detection that replaces manual user detection.

(Prevents said waste using current segmenting technique)

- when the "TTH in corruption message" feature is released.

- I may have this automated process do something extra like automatically restart download with new temporary file. (Optional or automated)

[Seeker 0]

Why are you pushing this issue? i without even reading the whole thing said straight up it wont be done, nor will the segmented be dropped. If i didnt know better dare i say you are spamming.

[Excelsium 0]

:thumbsup:

[Big Muscle 1]

btw to your solution... (deleting file from queue and starting it again) I don't see any benefit in it. How can it avoid wasting???

a) current solution - 1000x redownloading corrupted chunks

:thumbsup: your solution - 10x redownloading corrupted chunks + 1000x redownloading whole file

Your solution will do much more waste than current solution. But according to your writing, I think won't understand it.

You are sitting in this forum for whole day. You should fix the bug in your computer instead of forcing us to fix some non-existing bug.

so again NO AUTOMATED SOLUTION WILL BE IMPLEMENTED!!! DO YOU UNDERSTAND IT???

I think that you want only to call attention to yourself. I have never seen anyone who made 4 topics about same "problem" and wrote 5 pages of still same sentences without understanding what everyone writes to him. I don't think that you can't understand it, problem is you don't want to understand it.

OK. Just different situation. Imagine you wouldn't have a bug in your computer and you wouldn't have corrupted files. Would you write to this forum about this problem??? I don't think so. SO STOP SPAMMING THIS FORUM IF YOU AREN'T ABLE TO FIX YOUR COMPUTER!!!

[Excelsium 0]

Why are you pushing this issue? i without even reading the whole thing said straight up it wont be done, nor will the segmented be dropped. If i didnt know better dare i say you are spamming.

- I'm not trying to "push this issue" anymore, I will patiently await Big Muscles feature release.

1. I dont want the segmenting feature to be dropped.

2. Big Muscle has said he is going to add the TTH to the corruption message.

2.1. Hence I can build my own solution or have someone do it for me.

and share it with others if they require it.

I do not want to grow this thread anymore.

If you need a defensive type reply from me.. chances are its already in here.

Replies to Big Muscle

Defence posted already.

You can have the last post in the thread if you wish. Obviously my reply will probably be defence posted already.

This is the last post of mine - any more defences will be posted in this post so that it will not bump the thread anymore.

Also anything extra I have to say will be posted in this post as to not bump the thread.

Big Muscle has already offered me what I'm looking for - Thanks Big Muscle.

I can share my solutions with others as they require it.

- That auto delete and re add to queue is just one of them a Possible extra feature

to the main most important feature - automating the detection proccess which is currently manual - done by the user.

My techniques are enabled by the TTH being in the corruption message.

- I will add a name (or at least TTH) of corrupted file to "corruption detected" message.

and that's all.

I guess this is satisfactory since the user can open log and copy/paste > search for tth in queue.xml

but if possible it should be made user friendly.

It would be trivial to build something to notice a corruption message with same TTH is spamming the log and do something automated.. e.g. popup dialog box, pause download with that TTH in queue.xml or even delete and re add to queue. - maybe I'll do it myself if I cant find someone to do it.

[Big Muscle 1]

automating the detection proccess which is currently manual - done by the user.

no, it's not. Corrupted chunks are detected and redownloaded automatically.

to some moderator - I think that this thread can be locked because it has already been solved on the first page.

Summary:

- I will try ti fox the crash

- I will add TTH to "Corruption detected" message

- other solution written by Excelsium won't be done, because it would cause much waste - for example redownloading whole file instead of corrupted parts only.

- Excelsium is still writing the same words and he can't understand the answers, so it would only fill the forum database.

I don't have time to still write the same answers. There's better things to improve in StrongDC++ => currently I'm optimizing userlist memory usage :thumbsup:

[Zlobomir 3]

Before this will be locked, I wold like to use the chance to mention several things:

1. Thanks to Excelsium for trying to help us improve the program. We, at least I, appreciate the effort.

2. Thanks to BM for explaining over and over and over and... I am not a coder, so I can't reach the very depth of the explanation however.

3. As not being coder, I am trying not to argue with one repeatedly, at least from some respect...

4. BM has admitted up to now quite many bugs (f. ex. the emotions memory leak), so such flaming about covering bugs, is imho, say the least, abusive.

5. The proposed by Excelsium "fix" can do as much harm as good (missing the chance to complete a file), so I don't see any point in implementing it.

[Lee 84]

Will leave this open for further research if Big Muscle requires it, same for our staff. No more abuse.

[Excelsium 0]

- I have broken my agreement to not 'bump' - this was a stupid thing to agree too.

- I've moved my reply to Big Muscle below his reply by posting here.

- If people post more replies I will reply in new posts - because its easier for me to manage and easier for me and others to read.

- I dont care if people think I am spamming or trying to 'push this topic'.

- Topic can be locked or deleted - also dont care - does not affect me personally.

- On flaming? Disagree: I'm criticizing, pointing out facts, offering suggestions.

- A segmented downloading technique should be used in most of the dc++ network to spread the upload bandwidth load, etc etc.

automating the detection proccess which is currently manual - done by the user.

no, it's not. Corrupted chunks are detected and redownloaded automatically.

Quotes me out of context to what I was reffering to

Quotes from my opening post.

*2*. The user is not informed which temporary file is corrupted beyond repair by itself the client (StrongDC++).

*2*. Continually observing the transfers window and making a *Guess* which download(s) is/are not going to complete successfully due to a corrupted temporary file for that download that is corrupted beyond repair by itself the client (StrongDC++).

This is the manual process I was referring to and will automate.

- the other solution he is reffering to is only an addition to this automated detection technique - a feature I may require and will be made by myself or someone else, clearly not by Big Muscle.

He ignores my automated technique and just complains about a possible addition I may or may not add to my automated technique.

- Of course the user can decide to enable or disable any one of these deciding whether THEY think they are required for them personally.

- On the chance of missing to complete a file - well there are times obviously when the download will never complete. - When the corrupted temporary file is beyond repair by the client itself (StrongDC++)

- It could be set to a high number such as if the same TTH corruption message spams the log 200 times.

- I'm reffering to my techniques which I'm not asking Big Muscle to build anymore - I'm just continuing to discuss/defend them.

- except of course TTH in the corruption message which will be built by him as he has said - this is a necessary addition that enables the rest of my techniques to be built.

[Greg 0]

- I have broken my agreement to not 'bump' - this was a stupid thing to agree too.

- I've moved my reply to Big Muscle below his reply by posting here.

- If people post more replies I will reply in new posts - because its easier for me to manage and easier for me and others to read.

- I dont care if people think I am spamming or trying to 'push this topic'.

- Topic can be locked or deleted - also dont care - does not affect me personally.

- On flaming? Disagree: I'm criticizing, pointing out facts, offering suggestions.

- A segmented downloading technique should be used in most of the dc++ network to spread the upload bandwidth load, etc etc.

Quotes me out of context to what I was reffering to

Maybe you're just being passionate or whatever about this issue, yet it comes across as you're being overly defensive. This forum purely exists for users to inform us of bugs which we'll do our best to solve, and to post crash reports when availble. When a user, in future, can a related issue and searches and finds this thread, I doubt they will be impressed about constant defending/arguing between people. Lee did not leave this thread open for people to just keep argung/defending themselves, take it to PM or something if it's that much of a concern. However, Lee did have legitimate reasons for leaving the thread open, so it shall remain open for the time being.

[Big Muscle 1]

just a definition:

A software bug is an error, flaw, mistake, failure, or fault in a computer program that prevents it from behaving as intended.

--------

Excelsium is still trying to refer to some bug which (according to "bug" definition) doesn't exist. Everything in segmented downloading (except the crash), including the corruption definition, works as intended.

Yipee! I've manage to decrease memory used by userlist (10000 users in hub took about 30 MB before and now it's about 23 MB) :P