Segmented downloading techniques

[Excelsium 0]

To Greg: I feel that in this case full defence is required, so that people dont get the wrong Idea of what I am saying - also for my own reference, defence for things like an out of context quote that I think tears my thread to shreds.

If such things are not posted I will not have to make such defences.

Since most possible defences have been posted, I can make a short reply and refer them to existing defences.

I've said I will defend against/reply to every post someone makes that means

I will reply to everyones post if I feel a reply is needed.

Personally I think Its unfortunate if the thread is locked before my reply to the last reply I think needs a reply is posted but thats the way it goes.

I can of course save that in a private document.

I have this thread saved to disk.

- I'm using this thread for personal reference.

- If it becomes a problem it can be locked/whatever.

[Excelsium 0]

just a definition:

A software bug is an error, flaw, mistake, failure, or fault in a computer program that prevents it from behaving as intended.

--------

Excelsium is still trying to refer to some bug which (according to "bug" definition) doesn't exist. Everything in segmented downloading (except the crash), including the corruption definition, works as intended.

In this case I'm defining bug as gaps in techniques/lack of techniques that leads to said waste that can be prevented with new techniques or additions to existing techniques.

Perhaps I will remove bug from my opening post or include my definition of it.

- Technically maybe this could be in the feature request/feature discussion forum.

- Then maybe everyone will be happy.

- The thread was originally in the feature request forum I guess it was a mistake to have it moved because it does include a (non related?) crash bug.

[Big Muscle 1]

In this case I'm defining bug as gaps in techniques/lack of techniques that leads to said waste that can be prevented with new techniques or additions to existing techniques.

It's not possible to specify your own definition. Bug definition has already been established and noone (even you) can change it.

Btw I can't understand that you don't understand that your solution will bring much more waste than current solution :P

[Excelsium 0]

It's not possible to specify your own definition. Bug definition has already been established and noone (even you) can change it.

Btw I can't understand that you don't understand that your solution will bring much more waste than current solution

Yes it is possible to do this:

For ease of writing this I have defined bug to mean several things.

- "problem, issue, lack of technique, gap in technique"

I could replace bug with "cookiemonster"

I could replace all the instances of the word bug and replace it with "cookiemonster"

and everyone who read my definition of a cookiemonster would understand what I meant.

I will change at least change the titles in the opening post - everything else would greatly increase the word count. Remember this is an anal issue of Politics and so forth.

Current situation

Situation a. download will never complete in reasonable time.

* Download with corrupted temporary file that is corrupted beyond repair by the client itself (StrongDC++)

* beyond repair means it would take weeks or would never fix it. even going at 10 megabits a second and even if the download is small e.g. 50 megabytes.

My solution

- My solution would bring less waste.

Solution 1. download is deleted or paused preventing waste.

- If it is paused the user could "fix their machine" and come back and resume download.

Solution 2. download has a chance of completing in much less time. by automatically restarting the download with a new temporary file.

How could it possibly increase waste more than the current situation of it download the same chunk ad infinitum?

At the moment the user has to manually detect it and re add the download item to the queue themselves.

- Exactly what the automated technique would do.

If solution 2. turns out to be problematic

I would just stick with solution 1.

a) Present the user with dialog box with which download queue item has corrupted temporary file that is almost certainly beyond repair by the client itself (StrongDC++)

Pause the download allowing the user to "fix their machine" and come back later to resume the download.

[Big Muscle 1]

Situation b2. download has a chance of completing in much less time. by automatically restarting the download with a new temporary file.

WHY YOU CAN'T UNDERSTAND WHAT I WROTE TO YOU ???????

If you restart complete file it will be much more waste than only restarting corrupted chunk!!! Now you understand it????

The probability that new redownloaded file will be OK is same as redownloaded corrupted chunk will be OK!!!! Now you understand it????

The probability that redownloaded corrupted chunk will be again corrupted is same as new redownloaded file will be corrupted!!!! Now you understand it????

NOW YOU UNDERSTAND IT???

Test whether you understand it -

Which of following option comes with more waste:

a) you download 40MB. Chunk from 39-40 MB is corrupted. So we redownload this 1 MB. But this redownloaded chunk is corrupted again,so redownload again and again and again etc. (take we must redownload 1000x). So 1MB x 1000 = with redownloading we wasted 1000MB (= 1GB)!

you download 40MB. Chunk from 39-40MB is corrupted. So we redownload this 1 MB and again corrupted. According to your solution we try it 10x (value settable by user). So we waste 1MB x 10 (=10 MB) and after these 10 tries we redowload this file completely, but it will be corrupted again and again and again. (take we must redownload 1000x). So 40MB x 1000 = we wasted 40000MB (= 40GB) + 10 MB in tries to redownload only 1 chunk. So we waste over 40GB!

Select A or B according which option brings less waste.

[Excelsium 0]

Please read my post again and read solution 1. and also solution 2.

I have modified it alot since you posted your reply and before I read your reply.

I will repost a little here:

If solution 2. turns out to be problematic

I would just stick with solution 1.

a) Present the user with dialog box with which download queue item has corrupted temporary file that is almost certainly beyond repair by the client itself (StrongDC++)

b.) Pause the download allowing the user to "fix their machine" and come back later to resume the download.

Reply to your post:

About solution 2.

Yes that would bring more waste if it wasn't for the fact that 95-99% (or higher) percentage of the users other downloads complete successfully

- So this new restarted download will probably also succeed 95-99% (or higher).

- The file would not have to be redownloaded 1000x like the chunk.. the chunk redownload may be infinite

as said on a download going at full speed for a week at 10 megabits.

- The file would probably succeed on the first redownload - 95-99%+

- Please point out to me how this would waste more.

Please test your own understanding.

1. Chunk is redownloaded 1000s of times or infinitely

2. 95-99%+ of other users downloads complete successfully

3. Redownloading file with new temporary file has 95-99%+ success rate

The automated process (Redownloading file) would only kick in if the same chunk(s) have been redownload many times e.g. 200

making it almost certain the problem (redownloading chunk) is infinite or 1000s+

- Hence redownloading file wastes less and is currently a manual process that needs to be automated - it wastes the users time.

- If this (Solution 2.) turns out to be a dead end, can anyone argue for or against Solution 1.?

About Solution 1. Do you agree/disagree, why?

So on my solutions

2. Were arguing about the success rate of restarting download

- I say its high (95-99%)

- Big Muscle says its low (%)

- If it turns out the success rate is low then I will abandon this solution because it would cause even more waste as Big Muscle and others have said.

1. I have not recieved a reply from anyone on this solution yet. - it seems it has been ignored and all the replies have been about the controversial solution 2.

Solution 1. download is deleted or paused preventing waste.

- If it is paused/deleted the user could "fix their machine" and come back and resume/restart download.

a) Present the user with dialog box with which download queue item has corrupted temporary file that is almost certainly beyond repair by the client itself (StrongDC++)

b.) Pause the download allowing the user to "fix their machine" and come back later to resume the download.

- Dosn't have to be dialog box, can just be pausing and error in the error column.

[Big Muscle 1]

1. I have not recieved a reply from anyone on this solution yet.

I have already reply but you weren't able to understand. True, I haven't replied to solution 1 but to all solution generaly.

so again my answer THERE'S NO POINT IN OCCUPYING WITH ANY SOLUTION BECAUSE THERE'S ONLY SMALL AMOUNT OF USERS WITH CORRUPTED COMPUTER, SO THERE WILL NEVER BE ANYTHING MORE THAN REDOWNLOADING CORRUPTING CHUNK.

IF REDOWNLOADED CHUNK IS CORRUPTED AGAIN, IT'S NOT OUR BUSINESS!!! UNDERSTAND IT??? I WILL NOT SAY THE SAME THINGS STILL ROUND.

[Excelsium 0]

I have already reply but you weren't able to understand. True, I haven't replied to solution 1 but to all solution generaly.

so again my answer THERE'S NO POINT IN OCCUPYING WITH ANY SOLUTION BECAUSE THERE'S ONLY SMALL AMOUNT OF USERS WITH CORRUPTED COMPUTER, SO THERE WILL NEVER BE ANYTHING MORE THAN REDOWNLOADING CORRUPTING CHUNK.

IF REDOWNLOADED CHUNK IS CORRUPTED AGAIN, IT'S NOT OUR BUSINESS!!! UNDERSTAND IT??? I WILL NOT SAY THE SAME THINGS STILL ROUND.

Solution 1. is a seperate entity to Solution 2.

Big Muscle has not replied to Solution 1.

So I'm requesting replies about solution 1. that do not include solution 2.

I have already admitted I will abandon Solution 2. if the success rate is low.

- The redownloading chunk issue is your business - the user should be given solution 1. so that in the mean time that they have problems with their machine the client does not waste bandwidth.

- This also prevents wasting the users time - they don't have to observe the transfers window until they find the problem. (manual user detection).

[Big Muscle 1]

AGAIN, THERE'S NO POINT IN OCCUPYING WITH ANY SOLUTION, NO MATTER WHETHER IT'S SOLUTION 1, 2 OR 50000. DO YOU UNDERSTAND IT???

[Excelsium 0]

Solution 1. Is waiting for a reply.

I'm sure I'll get a good reply eventually agreeing or disagreeing.

If not from Big Muscle then from someone else.

Big Muscle said hes going to build the first part of 'Solution 1.' for me (TTH in corruption message).

I will build the rest of 'Solution 1.' myself as Big Muscle said he will not.

[Big Muscle 1]

LOL

nothing more to say here. It's like talking with robot which still say one phraseand can't understand what anyone says to him.

[Excelsium 0]

I don't see any need to further discuss this with Big Muscle.

It seems the only replies to Big Muscle would be to defend myself.

- I can see potential for massive shrinkage of this thread as 'Solution 2.' is indeed ridiculously wasteful if its proven to have a low success rate. It seems Big Muscle won't reply on 'Solution 1.'

- I will rework the opening post.

- I'd like to focus this thread on 'Solution 1.'

[Big Muscle 1]

It seems Big Muscle won't reply on 'Solution 1.'

AGAIN, THERE'S NO POINT IN OCCUPYING WITH ANY SOLUTION.

UNDERSTAND IT???

AGAIN, THERE'S NO POINT IN OCCUPYING WITH SOLUTION 1.

UNDERSTAND IT???

[iceman50 0]

so let me ask you this .... what if it is 1 Kilobyte of corrupted ram causing the bug..... you would never know (and surely never think to use memcheck to find the problem) and even if not that.... it could literally be an exponential amount of other problems causing the corruption ... .you would never know ... so no solution is fit to fix them

[Excelsium 0]

so let me ask you this .... what if it is 1 Kilobyte of corrupted ram causing the bug..... you would never know (and surely never think to use memcheck to find the problem) and even if not that.... it could literally be an exponential amount of other problems causing the corruption ... .you would never know ... so no solution is fit to fix them

- Its not a solution to fix the problem.

- Its a solution to properly inform the user of the problem so they can do something about it.

- It detects and stops the problematic process (The download with corrupted temporary file that is almost certainly beyond repair of the client itself (StrongDC++).

- It stops the problem in the mean time wasting their own and others bandwidth.

- It prevents wasting the users time by having them themselves detect the problematic process.

[iceman50 0]

yes but do you understand even informing the user is extremely futile for the exponential amount of problems it could be.... sure maybe it will stop wasted bandwidth ... but it is theoretically possible you could never find the probem so what solution do you want now? DON'T DOWNLOAD .... there is a solution for you it stops wasted bandwidth and countless retries that fail

[Excelsium 0]

yes but do you understand even informing the user is extremely futile for the exponential amount of problems it could be.... sure maybe it will stop wasted bandwidth ... but it is theoretically possible you could never find the probem so what solution do you want now? DON'T DOWNLOAD .... there is a solution for you it stops wasted bandwidth and countless retries that fail

- In that case the user should still be informed

As is the case I and others continue to use the client with "bugs in our machines" because of its 95-99%+ success rate.

I forgot a key point.

- For dealing with the 1-5% failures I will download them in normal mode in another instance of dc++ - this mode allows corruption. - if solution 2. is proven to have a low success rate.

- Where the files are available on bittorrent/etc I and other users catch the failures in this manner.

My current solution

Download everything in normal mode - I dont have time for manual detection

I'm testing an old version of Apex that may allow corruption in segmenting mode.

My old solution

Download everything in segmenting mode

- Detect failures manually

- Download failures in another instance of dc++ in normal mode or other p2p client.

[iceman50 0]

- Where the files are available on bittorrent/etc I and other users catch the failures in this manner.

What do you mean by this....i have created some solutions for this problem (in pseudo code and in actual raw c++) so i will develop my own test situations on a virtual machine

[Excelsium 0]

What do you mean by this....i have created some solutions for this problem (in pseudo code and in actual raw c++) so i will develop my own test situations on a virtual machine

I mean we download the files that failed to complete using sdc++ segmenting mode in other p2p clients.

- Or DC++ normal mode, where the files are only available on the DC++ network.

Thats my old solution though, I guess others still continue to do manual detection which is a pain for them.

Basically the detection should be automated... so we can catch the failures conveniently - add downloads to queue in normal mode or other p2p client.

[iceman50 0]

what BigMuscle was saying was there are not enough experiencing the problem and really no easy way to figure out what is causing it... the spectrum is too broad and it's just one of those things you will have to live with

[Excelsium 0]

iceman50: Have you just said you built some of the information solution I'm looking for ??

Sorry for misunderstanding..

what BigMuscle was saying was there are not enough experiencing the problem and really no easy way to figure out what is causing it... the spectrum is too broad and it's just one of those things you will have to live with

So hes said hes not going to build the simple 'informing the user about the problem' solution - I will or will have a friend do it it for me.

Or maybe someone else will

- As said hes already going to build part of it - TTH in the corruption message.. first enabling part.

[iceman50 0]

just a theoretical solution nothing to be 100% fix but it is possible maybe, i just found a new connection fix for DC++ 0/699 with the sockets causing problems under windows

[Big Muscle 1]

I mean we download the files that failed to complete using sdc++ segmenting mode in other p2p clients.

- Or DC++ normal mode, where the files are only available on the DC++ network.

heh and shouldn't we send corrupted files to Microsoft, they will fix them and send it to us back??? (just a joke, but same kind of solution as you have)

I don't know why you are stll talking about some failure in segmented mode. I said you many times that it has nothing to do with segmented downloading, but you aren't able to accept it.

So why are you still talking about segmented downloading???

Why don't you make a solution which will also work if normal downloading fails etc.?

I'm again telling that this corruption has nothing to do with segmented downloading. And also it has nothing to do with any downloading. The corruption was "created" after the file had already been saved to disk and it's unable to detect which download method had been used. If you had copy the file using standard windows copy mechanism to same disk area (if it's HDD error) then there would be a corruption too and Microsoft wouldn't be interested that your file was corrupted after using windows' copying.

If the corruption had been caused by Strong/Apex then we could talk about some solution. But the corruption isn't caused by Strong/Apex, neither in communicating with remote client, so there's no reason why we should implement some addition code to Strong/Apex.

Please read this thread at least 3x before you write anything more to this topic.

[Big Muscle 1]

If couldn't understand it, I'm trying to explain it in non-computer words.

There is some big hole on the road. You will drive car A and fall to the hole. Your solution is to use car B. You will fall to the hole too! Now understand that your solution is rubbish?

[Excelsium 0]

Possibilities for Big Muscles situation

-Cannot understand me or why my solution is necessary:

- Informing the user about the problem, not fixing the problem

(Properly Informing the user of exactly which downloads are failing and so on)

- dont want to repost whole thing.

-Cannot understand the problem.

-Trying to insult me or ruin the thread.. or some such.

- Does not want to build the rest of the feature

- Already said that I or someone else will build said feature/solution.

- Is joking around.

(another of those defence posts...)

I think that covers everything.. maybe wont have to post more defences.