Programming language ecosystem policies are mostly divergent, with only a few categories unified
The claim
The authors assert that PLE policies generally diverge except for malware, IP, and content removal categories
Published as: “The Policies of Programming Language Ecosystems: A "Qualitative" Analysis”
So what
The heterogeneity of PLE policies may impede the development of unified supply-chain security tools and suggests a need for more standardized, enforceable policy frameworks.
The five-minute version
The study finds that policies across eight major programming language ecosystems diverge rather than converge, with only a few generic categories (e.g., property rights, malware) shared by all. Inter-rater reliability was moderate overall (mean Fleiss’ kappa moderate) but varied widely across policy categories, reflecting ambiguity in many policies. The authors provide a qualitative coding of 38 policy categories and make their data and analysis scripts publicly available.
What they found 2
- Policies of the eight examined PLEs are overall divergent rather than convergent.
“Overall, the results indicate divergence rather than convergence between the policies of the eight PLEs examined.”
- Inter-rater reliability for policy coding is moderate on average.
“The mean and median of the Fleiss’ kappa coefficients are around values for which an adjective moderate has been seen as appropriate [41].”
What undercuts it the paper’s own results
- Perfect agreement was observed for five policy items, indicating some convergence: "There are also five cases with perfect agreement according to both inter-rater reliability measures. These are the content removal, account suspension, and account termination categories in Q4 and the property rights and binary categories in Q6."
- Four policy categories appear in all eight ecosystems, showing limited universal agreement: "Despite overall divergence, there are still four categories that are mentioned in all policies according to majority votes. These are: Q6(20), Q6(1), Q5(1), and Q4(2)."
What’s new here 4
- First systematic qualitative analysis of policy documents across multiple programming language ecosystems.
- Construction of a 38-category coding scheme to compare policy coverage and content.
- Measurement of inter-rater reliability (percent agreement and Fleiss’ kappa) for policy coding.
- Identification of both convergence (four universally present categories) and divergence among ecosystem policies.
Artifacts & methods 4
- Qualitative content analysis of PLE policy documents
- Coding scheme of 38 policy categoriesprimitive
- Policy documents from eight programming language ecosystems (npm, PyPI, Maven, etc.)dataset
- Script to reproduce figuressystem