File creation date attribute in filelists

[Pedja 0]

What you are trying to do is see what is new in the file list since previous times you looked at that user.

I suggested this way to introduce what's new long time ago, but developers simply rejected it. I support this idea beacuse it does not ask for anu change in protocol, and it's 100% compatible with al DC++ clients, because it requires just imrpvement on ApexDC++ itself.

However, introducing date field in files.xml is als not bad idea. I realy do not understand people who are concerned about fiel length. Date information length is really minor compared to everything else contained in files.xml.

After all, we should ask that files.xml be compressed instead downloaded as plain text file. Compression would really cut down transfer duration.

[patch1 0]

I suggested this way to introduce what's new long time ago, but developers simply rejected it.

I suppose if you mostly use Hubs with 100s of users it would be of no real benefit, a good search capability would be more appropriate.

However the hub I use has 5-15 regular users and 2-3 prolific downloaders so knowing what is new would be really valuable for me. (Local hub traffic is unmetered with my ISP)

Some users are trying to do it by analysing filelists with an external program. Not really ideal but perhaps that is all we can do if ApexDC developers are not interested.

[Satan 1]

While the feature itself is a good idea. I like it.

I just DONT LIKE the 'bloat' it will cause. I don't fancy having 10's of 1000's of file list's in my C Drive, I keep it small for a reason, performance.

[patch1 0]

I just DONT LIKE the 'bloat' it will cause. I don't fancy having 10's of 1000's of file list's in my C Drive, I keep it small for a reason, performance.

Shouldn't really be that bad. The only filelists on your drive would be the recent subset of files you have explicitly chosen to download and view. Better than what occurs if you disable automatic filelist deletion in ApexDc (or most internet browsers temp folder).

Leaving the option to delete all filelists on exit is probably reasonable, that way users who just search for things they want can minimise hard disk waste.

But then again it is the functionality I want, not really that concerned about the particular method used. So if anyone can think of a better or different way of achieving the same end, speak up.

[Satan 1]

Well how about users that auto-check users?

Or in my case, I use the match queue feature quite frequently on 50 users or so, so for them alone I will have 500 filelists

As I said, good feature, but 'bloat'.

[patch1 0]

As I said, good feature, but 'bloat'.

mmm

Sorry I'm not explaining myself clearly.

I had assumed if the option to delete all filelists on exit was retained it would produce no more "bloat" than you currently experience. Of course if you selected that option then you would only be able to see user additions during a single session.

Satan

I really have no particular attachment to this way of doing it, but I just can't think of another way which retains DC compatibility. What approach would you prefer was taken to find & display recent additions to a users file list?

[Satan 1]

There isn't another way to do it, that's the problem

The only way DC allows it to be done will bloat the client.

[dmvn 0]

At least it will be useful in LANs, so this increases the popularity of our favorite ApexDC in these networks. If we really think about 'bloating' then we should immediately remove all XML filelists and use a kind of binary format. The amount of filelists is the nearly-zeroth part of Network content, so if we care about a few kilobytes, then we should forget about downloading gigs of media files. Hence, what we're doing in the P2P world then?!?!

[Satan 1]

Not quite DMVN, I find that having a gig or 2 of media is a good use of hdd space, however, using MB's needlessly for file list's is kinda pointless....

Also, it will breed users to question, 'why is my apex folder 1Gb in size?'.

[dmvn 0]

We can optimize this bahavior... For example we can strip those attributes from XML immediately after FL is downloaded (if we dont want to use this feature and do not want to store additional data)... So OPs can carry lots of FL without bloating (it only affects FL DL time, not a disk space)

Moreover, we can introduce a special filter function that is applied to each downloaded FL that removes garbage from it... For example, exclude filez by mask/etc... :)

[Big Muscle 1]

and FL DL time is the most precious thing !!!

[patch1 0]

and FL DL time is the most precious thing !!!

Agree

So if adding a date stamp (to entries in the file list) significantly increases the file size & and makes it incompatible, it probably is not the way to go.

That leave more intelligent manipulation of the file lists already downloaded, or giving up :)

[dmvn 0]

Well, we CANNOT spend 2-3 seconds while FL is downloaded... But hence we MUST SPEND lots of time extracting FL changes by own eyes, if we need that changes... Well, i'm giving up in arguing, it's useless.

[Crise 26]

Well, we CANNOT spend 2-3 seconds while FL is downloaded... But hence we MUST SPEND lots of time extracting FL changes by own eyes, if we need that changes... Well, i'm giving up in arguing, it's useless.

No one has been arguing, I believe the point in this filelist download time issue is that the remote user who downloads the filelist has no way of choosing which filelist he or she wants... and to some even that 2-3 seconds may matter (and it is not always 2-3 seconds, it all depends on share size -> filelist size and remote users line speed).

[patch1 0]

Methods of finding out what

[Big Muscle 1]

Well, we CANNOT spend 2-3 seconds while FL is downloaded...

It's not 2-3 seconds! I took filelist from random user in our hub. He has 52 059 files in his share. Download speed was 9.36 kB/s.

Let's count. Timestamp is 64-bit number, so it's has up 20 digits! OK, let's take only 10 digits, because reaching 64-bit limit is only dream. Then you have to store 2 quotation marks to XML and some attribute name. Let's use TimeStamp (= 9 characters). So, result is 10 + 2 + 9 = 21. Let's round it down to have nice number :-) So 20.

Now count 52 059 * 20 = 1041180. So filelist will be larger about 1 MB. Download speed was 9.36 kB/s. So it will take 109 seconds to download this additional data.

His filelist has 1.53 MB. It takes about 167 seconds to download. So, it will take 109 + 167 = 276 seconds (it's more than 4.5 minutes!!!) seconds to download his filelist with timestamps!!!

No, thank you, I don't want this feature!

Now I could start same counting with another random user. But his filelist isn't finished yet. It has 5.64 MB and speed is 1.11 kB/s!!! What will happen when timestamps will be in this filelist???

But hence we MUST SPEND lots of time extracting FL changes by own eyes, if we need that changes...

Not we. You! how many people need to check FL for changes???

[patch1 0]

Not we. You! how many people need to check FL for changes???

+1

I do regularly as we have a small hub for (un-metered) local sharing

BTW

I agree with the proceeding analysis though --> time stamp at the receiving end

Your test showed time stamping adds more than my estimates, suggesting file path is transmitted as incremental changes (compressed) and does not include hash results.

But now I'm confused, how does ApexDC show I already have the file without the 40 character hash codes?

[sharrychris 0]

Nice Stuff!!

[SMT 0]

Now count 52 059 * 20 = 1041180. So filelist will be larger about 1 MB. Download speed was 9.36 kB/s. So it will take 109 seconds to download this additional data.

His filelist has 1.53 MB. It takes about 167 seconds to download

you "forget" bzip2 compression ))

every file have 74 bytes minimum (filesize=0, filename with single char), 100 chars typical.

so, his filelist would be 52 059 * 100 = 5 205 900

with timestamps: 52 059 * 120 = 6 247 080, just 17% increase

but timestamps will be similar to each other and much more compressible, than TTH. so really it will be less overhead than 17%

[patch1 0]

you "forget" bzip2 compression

I suspect the main problem with sending date information is that would be a change to the server side, so not an improvement a client could achieve. Even if a server added it as well their would be a compatability problem as it is a communication protocol change.

For that reason I would prefer ApexDC implemented an automated way of presenting changes from older file lists. These file lists are already retained, during a session. They can already be retained (optionally) between sessions. What is needed is to present the user with a menu showing the time stamp of saved file lists, enabling us to readily display recent changes.

Edit

This drop down menu would ideally appear when displaying a file list, along side the "subtract list" and "match queue" buttons.

[en_dator 1]

adding more information to the filelists imho can be useful, if file creation date is most useful im not so sure though.

The new partial file lists is one way that more info can be included without creating those huge numbers that you talk about BM, as most downloads now likely only will take a single folder and not the entire file list?

Possible is one thing, whether it should be implemented is another all toghether though, as usual

/1d