Supply Chain Attack on Popular Machine Learning Package Exposed User Credentials

Incident Overview

In a recent security incident, a widely-used open source package for machine learning monitoring, element-data, was compromised. The malicious version, tagged as 0.23.3, was published to the Python Package Index and Docker image accounts, affecting users who downloaded and ran it. With over 1 million monthly downloads, the scope of potential exposure is significant.

How the Attack Worked

The attackers exploited a vulnerability in the developers' account workflow, gaining access to signing keys and other sensitive information. This allowed them to publish a malicious update under the legitimate package name. Once executed, the compromised version scanned the host system for valuable data, including user profiles, warehouse credentials, cloud provider keys, API tokens, and SSH keys.

Timeline and Discovery

The malicious version was released on a Friday and remained available for approximately 12 hours before being removed on Saturday. The developers confirmed that Elementary Cloud, the Elementary dbt package, and other CLI versions were not affected. Users who installed version 0.23.3 or used the corresponding Docker image are advised to treat all credentials accessible in that environment as potentially compromised.

What Users Should Do

If you have installed or run the affected version, immediate action is required:

• Rotate all credentials that were present on the system where the malicious package was executed.
• Review system logs for any unusual outbound connections or data exfiltration attempts.
• Update to a safe version of element-data (latest clean release) and verify integrity via checksums or signatures.
• Monitor for suspicious activity in associated accounts (cloud, API, SSH).

Lessons for Open Source Maintainers

This incident highlights critical vulnerabilities in open source distribution workflows. Key takeaways include:

1. Secure account workflows — Implement multi-factor authentication (MFA) and use hardware security keys for package publishing accounts. Limit the number of users with signing access.
2. Audit signing key storage — Signing keys should be stored in isolated, access-controlled environments (e.g., hardware security modules) rather than developer workstations.
3. Monitor for anomalous releases — Set up automated checks to detect unauthorized version changes or metadata modifications in package repositories.
4. Establish a rapid response plan — Have procedures in place to quickly revoke access, publish advisories, and revert malicious releases.

For a deeper look at securing open source workflows, see our guide on Best Practices for Package Signing. The broader open source community must remain vigilant against supply chain attacks, as attackers increasingly target trusted distribution channels.

Conclusion

The compromise of element-data serves as a stark reminder that even well-maintained open source projects can fall victim to sophisticated attacks. Users and developers alike must prioritize security at every stage of the software supply chain.