45 replies to this topic

[Mek]

Toast, on 02 September 2009 - 12:51 PM, said:

Its only been a couple of days since StrongDC++ and RSX++ had its release of DHT capable clients and the cloud is really growing fast in the initial test phase there was about 30-100 users using DHT now its up to 650 and growing fast.

So what is DHT ?

DHT stands for Distributed hash table, its a network node that lets users share without being on the same hub a much needed feature for Direct Connect to increase popularity for using trying to find files.

This feature is more know by Torrent users since its a feature has been used there for a couple of years now but it didn't start there it started with a old p2p app called  Napster.

The DHT system in StrongDC++ is a rewrite made by Big Muscle and the specs can be found at ADCPortal Wiki for developers that are interested in joining up and using it.

Now for users that are concerned about privacy matters, don't worry the feature could be turned off simply under advanced settings so that your not apart of the DHT network.

This will probably end up in Apex in the future so i decided to post about it now.

For more info Wikipedia page

Looks promising. I hope it will help us find rare files  

[Lee]

Does it look in private hubs?

[Big Muscle]

Being in private hub + DHT is absolutely the same as being in private hub + public hub ;-)

[RadoX]

This could be a real problem for hubs that use IP bot to ensure ppl that the dl is in its own country cos like in Iceland you have unlimited dl inside your country but have limited dl outside, so only way to fix this is to use the GeoIPcvs and set it to only dl from IS (Iceland) IPs.

[Lee]

Toast, on 03 September 2009 - 08:06 AM, said:

Then turn it off whats the problem ? dont u ppl read what i wrote ?

It should be off by default, otherwise you're violating the privacy of users without them even know about it. Or maybe when the application is launched, it explains that instead of your share being visible to the hubs you choose to connect to, it's now visible to everybody in a global network.

[Lee]

Also, does the client display that DHT is enabled so hubs have the option to disallow entry?

[Big Muscle]

DHT has nothing to do with hubs so there is no reason why it shouldn't be visible to them. Or do you also display to hubs that you are connected to hubs DCDev Priv, DCDev Pub and e.g. Bestofall hub? I don't think so.

[RadoX]

Sorry I misunderstood the function of it, somehow in my head I thought that if I had DHT enabled I would connect others to outside sources.
Still I think users should be aware of this function so thay don�?´t dl from other countries cos thay don�?´t know what it does. Otherwise I really like this function and Im all for it.

[Big Muscle]

DHT is mentioned in a changelog so users are awared of this function. If they don't read it, it's their problem and not ours.

[Ophite]

I have some severe misgivings about this feature.

This is a fundamental change in network design; this builds one big P2P network, not the small-worlds network DC has always been up until now. This is a VERY BIG switch in paradigm. It may violate pre-existing user expectations of the software.

This feature really must not be hidden away in a changelog, otherwise users may inadvertently be sharing things with the whole world that they only meant to share with a select few. Are you, Big Muscle, prepared to accept responsibility for exposing them to that potentially severe risk? There must be a clear off switch, and I would argue, it should be off by default in Apex, certainly for a while at least.

(Do all DC users want to share with everyone? If they did, why are they using a small-worlds network like DC and not the many existing wide P2P networks like public torrent sites, eMule, or Gnutella? Isn't being small-worlds one of DC's characteristic advantages?)

Note that even Bittorrent includes ways to let the trackers tell it that public DHT publication is undesirable, and this field is very commonly used in the wild, for example to allow trackers to restrict access to the general public! Where is the analog here? There isn't one, at all. A client who's on the DHT may leak any downloading or shared file to the public inadvertently, without even being aware of it.

Note that the client does announce to the hub how many other hubs it's on, and the hub is already free to make decisions based on that. The client should, therefore, announce to the hub that it is on the DHT���?�as that is, essentially, one really big, extremely well-known, trivially infiltratable public hub to which all filenames and/or file hashes are leaked via multi-search!���?�and allow the hub to make decisions based on that, too.

Please, make sure that the client announces in the BINF or SUP and in NMDC that the DHT is on or off, and do not allow this to be cheated or emulated out if on. This is a decision hub owners absolutely need to be in on. Otherwise the only other alternative for security purposes is for all private hubs to ban the client outright. I don't want to advise that, but I may have to if this is not addressed.

Further, I should like more information about the actual DHT implementation. What are its defenses against DDoS attacks or prefix-takeovers via a Sybil attack? Does it have any?

Edited by Ophite, 03 September 2009 - 05:50 PM.

[Ophite]

Big Muscle, on 03 September 2009 - 10:40 AM, said:

DHT has nothing to do with hubs so there is no reason why it shouldn't be visible to them. Or do you also display to hubs that you are connected to hubs DCDev Priv, DCDev Pub and e.g. Bestofall hub? I don't think so.

Specifically as addressed above, you do, in fact, tell them that, at least in terms of numbers. Hub count, op/registered/unregistered. Right there in the tag and BINF.

[Lotus]

I tend to agree with Ophite.

But if DHT is OFF by default then 99% of the population will not use it.

Imho, the correct way is when someone install StrongDC++, a pop-up window appears and explains him what is DHT, giving him the option to enable/disable it (!)

This window is configurated in such a way that if the user skips this window, not choosing anything or not reading, DHT will be set to ON.

And thank you BM for this huge step forward, imho.

[Big Muscle]

The main problem is that a lot of users aren't aware of the purpose of SHARING DC-based application. It has been designed to share files with users. And not to share files with ONLY SOME users. Hubs are only a mean to allow connecting to these users. The second thing is that neither protocols (neither ADC nor NMDC) nor whole application differenciate between public, private or somewhat other. There's only internet network. So, if users use application/protocol for some special case, it's only his turn to study all documentation (changelogs, forums etc.) and to customize app's settings to its need. So there is really no reason to have DHT defaultly off or to inform user with some special things. The third thing is that it is open source application and users take its own responsibility for using it.

If hubs will ban StrongDC++ due DHT, it's only their problem and I will only be glad that StrongDC++ users won't be in such hubs. StrongDC++ has been designed for sharing with all users, so our users shouldn't go to hubs which are against sharing.

And the last thing, if you don't want to share your files with all people, then turn your client off, delete it from your HDD and go away from sharing world.

[Lotus]

BM: you have a point too. After rethinking a bit, and considering other posts on this forum and your's, I think I understand now that SDC is meant for share and implemented this way. While, instead, some hub-owners are anti-share and anti-DC... so this is why SDC does not meet their needs all the time.

Indeed DHT is creating a virtual huge public "hub". But that hub is only meant for share. It has no chat, mainchat, op-list or something like regular hubs have... so the "DC spirit" is not broken even if this option is enabled by default without user knowing. Everything is the same as before, DC is the same, only download speed is boosted.

One particular effect of DHT, however, is that you no longer need to connect to big hubs to be able to download fast. This is a change. But this seems like a positive change from the user perspective. I don't know how hub owners would react to this, because I think it will somehow decrease hub's hegemony and increase DC decentralization. Hub X will not be so popular and attractive as before, because I will no longer need to connect to it to download fast. Because of DHT I can connect to any hub, even small, and still download fast... But again, it seems like a positive to me.

In my opinion DHT is much better then not DHT.

BM: some of us try to understand and just express our opinions. No need to be upset...

[Big Muscle]

Exactly, what you are saying is that what I wanted to avoid by not implementing keyword publishing in DHT. Therefore, connection to hub is still required to search files by keywords. DHT can be used only for TTH searches... also, there is another positive for DHT - when you are connected to hub and there's some user. When this user is found as a source (in hub or via DHT), it will initiate connection via DHT and not via hub. So as the result, it will decrease hub load.

[Ophite]

Big Muscle, on 03 September 2009 - 07:13 PM, said:

The main problem is that a lot of users aren't aware of the purpose of SHARING DC-based application. It has been designed to share files with users. And not to share files with ONLY SOME users. Hubs are only a mean to allow connecting to these users. The second thing is that neither protocols (neither ADC nor NMDC) nor whole application differenciate between public, private or somewhat other. There's only internet network. So, if users use application/protocol for some special case, it's only his turn to study all documentation (changelogs, forums etc.) and to customize app's settings to its need. So there is really no reason to have DHT defaultly off or to inform user with some special things. The third thing is that it is open source application and users take its own responsibility for using it.

If hubs will ban StrongDC++ due DHT, it's only their problem and I will only be glad that StrongDC++ users won't be in such hubs. StrongDC++ has been designed for sharing with all users, so our users shouldn't go to hubs which are against sharing.

And the last thing, if you don't want to share your files with all people, then turn your client off, delete it from your HDD and go away from sharing world.

I can't help but feel you fundamentally misunderstand the nature of a small-worlds network like DC, Big Muscle, where we have circles of friends with each other, and so-called "super-peers" connecting those circles. It's actually a time-honoured design for a reason���?�see the 2002 paper "The Darknet and the Future of Content Distribution"[1] by MS Research, citing the design of older networks and their advantages in a modern context (section 2.5).

What you are proposing is more like Overnet or eMule's KAD than anything else, in fact. But BM, if I wanted eMule, I'd use eMule. I wanted something else. Many chose DC because it gave them choice; where to connect, where to go, who to talk with, who to share with. Taking that choice away isn't in my view a step forward���?�it's a step back.

Are you absolutely certain you wish to introduce essentially a brand new big P2P network with no privacy, anonymity, encryption or security in this climate? Is that really a smart idea? Are you further absolutely certain you wish to make users of your client join this network when upgrading, without clearly and openly disclosing this to them? (Burying it in your changelog does not count.)

Are you aware of the security risk that this may pose? Have you analysed the DHT design with respect to DDoS attacks? It's based on single unencrypted unauthenticated UDP packets from what I can see... I do not think that is a good idea.

/O1

[1] http://crypto.stanfo...02/darknet5.doc