High CPU use in 1.1.0

[GregJ 0]

I've just upgraded to 1.1.0, and am now experiencing long periods of extremely high CPU usage for no apparent reason. I do have a fairly large download queue(150GB in 13,000 files), and quite a large shared file list(280GB in 48,000 files), so I'm wondering if it might have anything to do with either of these. Between the long peaks, there are short periods of normal CPU use, no longer than a minute or so. One thing that I've noticed is that the quieter periods appear to be associated with having fewer handles open, but this could just be coincidence.

Attached is an image from Process Explorer.

[GregJ 0]

OK, with further testing, this appears to be solely related to downloads, not uploads.

When logged onto no hubs, it doesn't happen.

Logged only onto a hub where sources for my downloads are, it immediately happens.

Logged only onto a hub where people are downloading from me, it doesn't happen.

After setting my entire download queue to "paused", logging back onto the hub where sources are, it no longer happens.

Now I know strict management of my download queue will control it, I don't even know if it's a bug at all. The only question I'm left with is why this never showed up for me in earlier versions.

[RoLex 1]

There are few things that may cause this.

1. As I have a fast line that is capable of transferring as maximum at 12,5 MB/s, and I have pretty slow computer. To solve that I need to set download speed limit or reduce number of simultaneous downloads, maybe disable multisource feature.

2. As I have a pretty slow computer, hashing of files makes my CPU overloaded. To solve that I need to set hash speed limit.

3. As you have a huge queue, it is being updated in real time. To solve that there was a setting once, I'm either blind or it has been removed, no idea how to solve that anymore.

Second thing isn't your case as I understood, check other two.

Does anyone know how to solve number three? By handing me the glasses? :(

[mcravener 0]

High CPU use has been my issue too - see my post from the 14th of June. Thanks for process logging GregJ.

It seems to me to have to do with when Apex starts up, possibly when files are downloaded directly from when the Apex program starts. Yesterday I did manage to keep 1.1.0 stable with reasonable CPU use (10 connected hubs) for several hours, even downloading 4 files at a total speed of 2 mbit/s, though these downloads were put in queue during the same session. Today when I started the Apex program with the same four files in queue, and a known available user to download from, one connected hub was enough to overload my processor.

Queue size has not affected CPU use. I suspected it might, so I reduced my queue from 60 to 30 files but CPU use still runs high.

Hashing has worked fine. I did it with no hubs online though.

[Satan 1]

Well Apex will wait your predefined search interval before starting a search so that cant be the prob. Dont think it is anyway.

If you have such a large queue that is the reason, unless these files are all huge and in essence the xml file is still small that is.

Queuing up thousand's of files will swamp the client as it is opening numerous connections at once.

[mcravener 0]

Well Apex will wait your predefined search interval before starting a search so that cant be the prob. Dont think it is anyway.

If you have such a large queue that is the reason, unless these files are all huge and in essence the xml file is still small that is.

Queuing up thousand's of files will swamp the client as it is opening numerous connections at once.

Well yes but this does not explain my CPU usage when running Apex. What GregJ and I have both noticed is that downloading may be connected to the high CPU use.

[mcravener 0]

I'm running 1.1.0 now but the download queue seems to be the cause of the drag on the CPU. With 5 files in queue and all paused but one, and this one file has 20 user sources but no downloading going on, Apex 1.1.0 is intermittently running on 2% to 84% of my CPU (see upper figure).

With all files in queue paused Apex 1.1.0 runs 2-10% of my CPU, with occasional spikes of 40% (see lower figure). The difference with the queue actively on and off is substantial and is producing quite different behavior from my experiences with Apex 1.0.1.

As a footnote the upload going on was 3 slots at a total of ca 260 kB/s.

[mcravener 0]

Source of bug found!

With Queue/Enable Multisource => Disabled - I get the buggy CPU use (and TTH inconsistencies after downloading for a while)

With Queue/Enable Multisource => Automatic - things are running very smoothly.

I can't say what caused GregJ's CPU problems but now all works well on my computer.

[RoLex 1]

Nice discovery. Looks like DC users are starting to completely forget single source functionality. Not many who uses that anymore. :)

[dmvn 0]

But app *must* run smoothly in all possible cases. Disabled multisource *cannot* eat much CPU time itself (moreover, it must *save* CPU time...), it's definitely a bug somewhere...

[Crise 26]

mcravener, could it be possible for you to try if ypur observation also holds true for StrongDC, as if so it would be best if this was fixed there... (besides BM knows the multisource code far better than f.ex. I do )

[mcravener 0]

Sure, which version of StrongDC? Or why don't I compare the StrongDC versions for both 1.0.1 and 1.1.0... :)

[Crise 26]

Testing against latest SDC is enough as 1.1.0 is based on it...