piwheels stats 2021Q2

Posted 4 November 2021 by Ben Nuttall

In the period of April – June 2021, 8,471,941 packages were downloaded from piwheels, bringing the total to 48,609,840. This has saved 38 years in the period, and 525 years in total! There were 56,200,613 pip searches, 764,031 project page hits, 386,472 web page hits and 116,486 project JSON downloads in the period.

We experienced a one-day spike of 168k downloads st the start of the period but averaged at 94k per day:

Downloads saved around between 11 and 14 years per month. It seems this is falling, presumably due to more downloads coming from faster platforms (Pi 3/4 vs 1/Zero).

The top 10 downloads were:

1. requests (153,175)
2. chardet (150,367)
3. numpy (140,902)
4. importlib-metadata (131,558)
5. octoprint-pisupport (124,520)
6. pip (118,897)
7. idna (109,821)
8. h11 (104,934)
9. urllib3 (99,970)
10. typing-extensions (99,920)

The top 10 pip searches were:

1. pip (8,729,381)
2. wheel (5,982,230)
3. setuptools (3,978,520)
4. pyrogram (1,502,564)
5. future (751,603)
6. youtube-dl (749,239)
7. aiohttp (696,422)
8. pyngrok (661,208)
9. aiodns (649,962)
10. six (547,745)

The top 10 project page hits were:

1. opencv-python (1,210)
2. opencv-contrib-python (1,108)
3. numpy (981)
4. scipy (700)
5. tensorflow (686)
6. qhub-jupyterhub-theme (493)
7. pillow (445)
8. cryptography (429)
9. pandas (381)
10. pip (335)

The top 10 project JSON downloads were:

1. awscrt (12,815)
2. numpy (220)
3. requirements-parser (91)
4. scipy (62)
5. opencv-python (58)
6. requests (46)
7. notion2alfred-snippet (43)
8. pyyaml (40)
9. devjohn06test (38)
10. opencv-contrib-python (37)

See the JSON API page for more information on using this API.

Platforms

Linux makes up over 99.9% of all searches so I’ve discounted the rest. Raspberry Pi OS and Debian are now combined (due to Raspberry Pi OS identifying itself as Debian). This group obviously dominates with nearly 99%. Ubuntu remains in third place with under 1%, although we don’t officially support it.

Buster accounts for nearly 94% of PiOS/Debian usage, Stretch accounts for just 6% and there’s practically no Jessie usage:

armv7l (Pi 2/3/4 platform) is still a majority architecture with 93% of searches from Arm devices (80% from armv7l). We now get more hits from aarch64 users than x86_64 and armv6l combined, when it previously trailed behind both. It seems usage of the 64-bit OS image is on the rise, although we do not as yet provide aarch64 wheels.

Python 3 has a supermajority of usage, and 3.7 (the version provided by the current stable distro) has the highest share with 71%. Python 2.7 comes second with 15%, followed by 3.8 and 3.9. Generally most users are using the Python provided by the current Debian stable – which is due to change this summer, so we’ll start to see a rise in Python 3.9 usage which could become the leader by the end of the year.

We’re also logging pip and setuptools versions:

Note that 18.1 is pre-installed in Buster, and 9.0.1 is pre-installed in Stretch. 20.1.1 is a recently released version, so likely the most common for people who update their pip.

Similarly, setuptools 40.8.0 is what’s pre-installed in Buster, but the version of pip in Stretch doesn’t send the setuptools version (33.1.1) in the user agent like it does in newer versions.

We see about 5TB of bandwidth each month, peaking at 5.35TB in April. The breakdown of bandwidth in the period is 84% pip downloads, 3% pip searches, a tiny amount (0.07%) of website traffic (project pages, JSON, etc) and the rest (13%) comprising mostly of wheel downloads from browsers (not pip):

And broken down by month:

Check out the source of this post in a Jupyter notebook: github.com/piwheels/stats/blob/master/2021q2.ipynb