Bug: Missing license information still not allowed in crates from a private registry

[kornelski]

Describe the bug

I've tried to configure it to ignore licenses from a private registry:

[licenses.private]
ignore = true
registries = [
 "http://private.example.com/"
#  "sparse+http://private.example.com/" # tried this too
]

but I'm still getting an error about crates not having a license:

error[unlicensed]: internal-only-crate = 1.1.1 is unlicensed
  ┌─ sparse+http://private.example.com/#internal-only-crate@1.1.1:2:9
  │
2 │ name = "internal-only-crate"
  │         ━━━━━━━━━━━ a valid license expression could not be retrieved for the crate
3 │ version = "1.1.1"
4 │ license = ""
  │            ─ license expression was not specified
  │
  ├ internal-only-crate v1.1.1

There is no license or license-file in Cargo.toml of that crate, on purpose. It's proprietary, and there's no SPDX identifier for that.

To reproduce

Publish a crate to a private registry (I've used freighter), use it in a workspace, and run cargo deny check.

cargo-deny version

cargo-deny 0.16.3

What OS were you running cargo-deny on?

MacOS

Additional context

No response

[jongiddy]

The check for private licenses uses the matches_url method that has a bug fixed by #739

Does cargo-deny built from main fix this problem for you?

[kornelski]

I've tried a build from the latest commit f6e40d8

ignore-sources = [
   "git+ssh://git@internal.git/internal_package1.git"
]

worked for me to fix this case:

cargo deny check licenses
error[unlicensed]: internal_package1 = 0.1.0 is unlicensed
  ┌─ git+ssh://git@internal.git/internal_package1.git?branch=main#internal_package1@0.1.0:2:9
  │
2 │ name = "internal_package1"
  │         ━━━━━━━━━━━━━ a valid license expression could not be retrieved for the crate
3 │ version = "0.1.0"
4 │ license = ""
  │            ─ license expression was not specified
  │

but I can't figure out how to allow private packages from the current workspace and a private registry. I've tried all the combinations of registry URLs in licenses.private.registries and licenses.private.ignore-sources.

error[unlicensed]: internal_package2 = 2024.12.5 is unlicensed
  ┌─ path+file:///Users/me/internal_package2#2024.12.5:2:9
  │
2 │ name = "internal_package2"
  │         ━━━━━ a valid license expression could not be retrieved for the crate
3 │ version = "2024.12.5"
4 │ license = ""
  │            ─ license expression was not specified
  │

error[unlicensed]: internal_package3 = 3.2.0 is unlicensed
  ┌─ sparse+https://internal.registry/index/#internal_package3@3.2.0:2:9
  │
2 │ name = "internal_package3"
  │         ━━━━━━━━━ a valid license expression could not be retrieved for the crate
3 │ version = "3.2.0"
4 │ license = ""
  │            ─ license expression was not specified
  │

BTW, the error shows license = "", but the license field is completely absent in Cargo.toml of these packages.

[williamareynolds]

I found that if the private package's manifest doesn't contain the publish key, I couldn't use licenses.private.registries. The package had to have a matching url in publish or publish must be set to false i.e.

# Cargo.toml in my private package
[package]
# ...
publish = [
  "my.private.registry.com"
]
# ...

with "my.private.registry.com" added in licenses.private.registries in deny.

OR

# Cargo.toml in my private package
[package]
# ...
publish = false
# ...

Then in deny you don't have to do anything other than setting licenses.private.ignore = true.

In other words, licenses.private.registries is selecting for packages whose manifest specifies that registry in package.publish. As long as licenses.private.ignore = true, packages whose manifest specifies package.publish = false will have their license ignored as well.

licenses.private.ignore-sources seems to be primarily interested in where the local copy of the package was actually obtained from. Unfortunately, it has a bug that I'll file separately from this one where it seems to strip the ".git" from the end of my configured URL. As a result, having licenses.private.ignore-sources = ["https://my.registry.com/index/package.git"] will only allow packages sourced from "https://my.registry.com/index/package". Without the ".git" on the end it doesn't match, so I can't use it.

If someone can confirm that my understanding of everything here is correct, I'd be happy to make a doc contribution with some clear examples of how each of these is meant to be used and create that other bug (if it is indeed a bug).

[AlistairEpic]

I was having a similar issue with sparse indexes, after digging in a little I found that when adding the urls from ignore-sources it was runing through the crate::normalize_git_url(&mut url) function (seen here) regardless of if it was git or not, and this function was removing the trailing / which broke the comparison.

As a workaround I added a second trailing / so after it did the trimming there was still one left

e.g.

ignore-sources = [
   "https://internal.registry/index//"
]