Skype ideas and experience -> Apex DC

[BigLoad8TB 0]

I am running very big 5 P2P sites (Verlihub based), 5000+ active users, more than 60000 registered. The DC++ was choosen to replace FTP file sharing server. So the content was shared via donor users who share 7TB of data or more via DC++ clients instead of FTP. User surveys show that about 60% of users have difficulties connecting and about 30% finally failing to connect and abandon Apex DC or other DC++ based program because it is too complex to connect. Many ask from me: "Could you do that like Skype does? No settings required!". This request seems more important to users than super seeding features etc. Very little users understand something about the ports, NATs etc. The biggest problem is that defaults rule the IT world and only small percentage of users are able to change the defaults. If you put passive mode as the default after prepackaged client installation, most users stay on passive and the hub is of no use as they cannot connect to each other. If you put active then 50% of users cannot get search working because they are either in offices under firewalls or at home behind NAT with 192.168.. address. Lets be more technical, this algorithm is similar to how Skype provide universal setup-free connectivity, and this could be ideal for DC++ to establish "AUTOMATIC" mode for connection settings in which:

1. Separate but very simple in principle web-service style function must be established with the following functionality: client calls the service and provides port to test the TCP/UDP connection and to send some test data via both protocols. Service replies with clients external IP address (this is dynamic for most DSL lines and will be different at every connection), and then service tries to connect and send the data to that IP address to port number provided by client. If client receives this data, it means the incomming connections are working and its OK to work in active mode. If incoming address is internal well known IP, client can go to step 3 directly. Otherwise client must first test if incoming connection works to his IP address. Client could get this web service address from the hub on the initial list of info after connection, or it must be specified in options, and this can be provided/prepackaged to users by ISP.

2. If incoming connection is not working, then client must try incomming connection to fixed predetermined port, for example 1412, instead of specified ports. If connection is established and test data is received, connection problem is solved and program can work in active mode. This would allow to make easier instructions for port forwarding or firewall setup, and would allow to preset DSL routers NAT for DC++ by ISP.

3. Client tries UPnP. If this succeeds, problem is completely solved and this is the best way to provide external listening TCP and UDP ports. Hovewer 80% cases UPnP fails because the default in the modems usually is that UPnP is disabled, or modem does not support UPnP. If UPnP fails:

4. Client tries to connect to SOCKS 5 proxy server, if it is provided in settings OR, very important, if it is provided by the HUB with login username and password, maybe on login text in chat just after connecting. As an ISP I would install several socks5 servers to support passive users. Easy to do, standard Linux and Socks 5 server software, and most users behind office firewalls would be able to run in active mode.

5. If the socks server reached maximum users or not responding, or if there is no socks server provided in options and no proxy info from the hub, client displays a warning to the user and goes into passive mode.

To say it shortly, it is very important that client AUTOMATICALLY detect its external IP address and must do all the possible steps to establish active connection in this proposed order:

1. Direct connection via specified port number.

2. Direct connection via standard port number (1412)

3. UPnP

4. Outgoing AND Incoming connection via SOCKS 5 proxy (works for any firewall, proxy can be reached via 80 port) and client can establish listening TCP/UDP port on proxy server, packets then forwarded from proxy to client.

5. If everything fails - then a warning must be displayed to the user, and client goes into passive mode. In this mode, if the user double-click on other passive user, warning message must be shown instead of trying to connect and failing when this is 100% impossible. Passive client shall not try to connect to any other passive client in any case.

What other users asked in this forum is that some active users would do tranzit and allocate bandwith for two passive users. This is what Skype does to achieve universal connectivity. I believe this is impractical for P2P as it is too complex to implement and also users have very limited uplink on DSL lines, uplink is fully occupied by uploads and there is no space left for transit. So Socks proxy installed by ISP is much simpler and high bandwith solution (todays server for 3000EUR can easily handle up to 2Gbit/s proxying as no slow discs are required for proxy function, two passive users will just be short-connected in high speed socks server memory). If any Apex DC developers reading this, please consider this proposal. No connection settings is why Skype is so popular - "works behind any firewall" just after install. Must be same easy with DC++. Thank you.

[Zlobomir 3]

Yes, the one way around is to simplify and improve the programs. The other one is to educate the users. I still believe that the second is just more in DC++ spirit. There are guides and stuff...

Or we need just a small code, which to "press" Check Settings on each startup and according to test results to set up the active/Passive mode?

There is already a request for users acting like proxies, even for unlimited no. of passive users, but with bandwidth they specify.

Edited September 27, 2006 by Zlobomir

[Lain_13 0]

Many users don't want read manuals or something other...

...I know at last 5 peoples, who don't want to do this...

...but they use DC...

...one of them call me every time, when they system crash or something goes wrong...

...I hate it - but it is.

I think in setting window with connection setting required long big button "Try automatic configure". Or contrast clickable text with this function.

(I think this is high priority task)

[poy 0]

1. Direct connection via specified port number.

2. Direct connection via standard port number (1412)

3. UPnP

4. Outgoing AND Incoming connection via SOCKS 5 proxy (works for any firewall, proxy can be reached via 80 port) and client can establish listening TCP/UDP port on proxy server, packets then forwarded from proxy to client.

5. If everything fails - then a warning must be displayed to the user, and client goes into passive mode.

there is one thing you have forgotten, and that is to my mind the most important: telling the user to stay in active mode, and giving him instructions on how to forward ports manually. because, in many cases, when both direct connection and UPnP fail, i think a Socks5 proxy isn't really a good solution, and it would be better to have the user manually forward the right ports.

[CrazyBerserk 0]

Many users don't want read manuals or something other...

...I know at last 5 peoples, who don't want to do this...

...but they use DC...

...one of them call me every time, when they system crash or something goes wrong...

...I hate it - but it is.

The USERS want the files, THEY are the ones that are demanding. So IMO THEY have to work a little on that, reading the how-tos, having a look at the loads of info and help sites the community, the mainchat and many hub owners and their OPs offer them. Not everything should be "pre-chewed" for everyone.

So I think, the client should not be totally plug and play. DC isn't Skype. And I'm glad that it isn't, actually. Using one's brain and actually attending to DC and understanding a bit of DC's spirit prior to using it just for downloading might be the better way for the community. But maybe that's just me...

[Zlobomir 3]

No, IMHO you are right. Missing AI and no I at all are equal, >> if no intellect, people are useless. Even now there are pretty enough "zombies" asking "uuu, what does this button do" (pwrd by Dexters Lab) in the MC.

[Lain_13 0]

Hmm... maybe you right...

I hate st**id questions... but I try always answer on them...

...it's my nature.

[Zlobomir 3]

I also answer all questions I'm capable. Mostly by typing a link to DC help places. :)

[BigLoad8TB 0]

If all users would be like people write they are, then everyone would be nice and tall in this world. Who would want to be fat, when you just need to eat a little bit less to be in very good shape. Who would smoke knowing that this causes a cancer - just read a little bit about health and drop that cigarette.

Welcome back to reality, guys.

I believe even you will have much more available content via DC you like, if there are DC tools that make it EASY to share. Most people will NOT read manuals, they simply want to share, not to read manuals. Try to release both autoconfiguring client and manually configured like today, and I can quess easily which one will be multiplied in millions and which will die.

And I am also loosing the idea of creating Apex DC++ if the logic is as posted in answers above, that users must figure themselves how to setup ports etc. There is original DC client, there is even Strong DC - why care and improve even more.

I simply posted what are the biggest problems for the users. 40% cannot manage themselves to establish active mode, and half of them do not understand what is it and why they need to take care about such a deep technical things, especially when other pograms do that for them. If you leave default in passive mode then nobody can share and community dies. If you leave default in active, most of chat content is phrases from these 40% annoyed users that search does not work and they cannot use anything except chat to complain about the bad program.

So if anything can be done to help these 40% of users, its worth the effort, because we will have thousands of users more in active mode. That means bigger communities and more content for everyone.

[Zlobomir 3]

IMHO quantity is not equal to quality. As poy said above, some settings just have to be made manually. If not, the program will just rely on some "working trash". Like the really good, but ultraslow file transfer in Skype. There is something to help these 40% of users - a real live support hub in Favs. Excuse me, but if someone thinks that this is not sufficient, I would prefer him out of DC. Because his share will be out of order and files will be broken. What files btw? The first 1GB of private documents or XXX or whatever you have? Or the first 500 found files by the auto-search? As at least a semi-educated user, I would be annoyed by such a program immediately, together with other 60% of the users.

However sure there are thing that may be improved... For example addition (with prompting for user) of the 5 best hubs (acc. tto some list) and/or another 5 for the specific country (acc. to GeoIP). And a better and more complete /help. No hard feelings, just frank opinion. :D

[CrazyBerserk 0]

Most people will NOT read manuals, they simply want to share, not to read manuals.

According to my experience, most users are in DC for downloading, not for sharing. Sharing for them is something that is necessary to be able to enter the hubs. (But well, as I already said at another place today, in our campus LAN DC island we cannot really know about the situation of DC in the internet.)

If you leave default in active, most of chat content is phrases from these 40% annoyed users that search does not work and they cannot use anything except chat to complain about the bad program.

Well, at least people then would in fact use DC for chatting, not as in case of my hub where more or less 70% of users use it just for leeching the stuff they want and go offline as soon as possible. And more than half of them doesn't even see a need to share the stuff they downloaded next time they're online.

BTW, standard reply to such complaints in my hub is something like 'enter +help and read before complaining' or 'enter +mode and come back if problem still exists'. (+mode contains quite a short text with the 3 or 4 most common reasons for active mode issues and their solutions.)

So, welcome to DC community, where in my opinion the chat indeed is an important part of.

What files btw? The first 1GB of private documents or XXX or whatever you have?

Word! How many new users can be found sharing their 'Windows', 'Program files' and/or 'Documents and Settings' folders in order to share at least a bit? Or because they don't know better?

As conclusion, I still think that it is important for not a bigger but a better DC community, if people have to use their brains a bit and get in touch with DC's spirit first. Though, I know that fewest users are nerdy computer cracks, anyway.

As Fox Mulder used to say: "The help is out there." :)