View all changes

opam 2.2.0~alpha

Feedback on this post is welcomed on Discuss!

We are happy to announce the alpha release of opam 2.2.0. It contains numerous fixes, enhancements, and updates; including much-improved Windows support, addressing one of the most important pain points identified by the OCaml community. You can view the full list of changes in the release note.

This alpha release is a significant milestone, brought together by Raja Boujbel after years of work from the opam dev team (Raja Boujbel, David Allsopp, Kate Deplaix, Louis Gesbert, in a united OCamlPro/Tarides collaboration) with the help of many community contributors. We also thank Jane Street for their continued sponsorship.

This version is an alpha, so we invite users to test it to spot previously unnoticed bugs and work towards a stable release.

Windows Support

Opam 2.2 comes with native Windows compatibility. You can now use opam from your preferred Windows terminal! We rely on the Cygwin UNIX-like environment for Windows as a compatibility layer, but it is possible for a package to generate native executables.

The main opam repository is not Windows compatible at the moment, but existing work on a compatible repository (originally from @fdopen) and 32/64 bit mingw-w64 packages (by @dra27) is in the process of being merged. Before the final release, we expect it to be possible to run opam init and use the main opam-repository for Windows.

How to Test opam on Windows

This alpha requires a preexisting Cygwin installation. Support for full management of a local Cygwin environment inside of opam (so that it's as transparent as possible) is queued already and should be available in 2.2.0~alpha2 as the default option.

  1. Check that you have all dependencies installed:
  • autoconf, make, patch, curl
  • MinGW compilers: mingw64-x86_64-gcc-g++, mingw64-i686-gcc-g++
  • Or if you want to use the MSVC port of OCaml, you'll need to install Visual Studio or Visual Studio Build Tools
  1. Download & extract the opam archive
  2. In the directory launch make cold
  3. A coffee later, you now have an opam executable!
  4. Start your preferred Windows terminal (cmd or PowerShell), and initialise opam with the Windows sunset repository:
  • opam init https://github.com/ocaml-opam/opam-repository-mingw

From here, you can try to install sunset repository packages. If any bug is found, please submit an issue. It will help opam repository maintainers to add Windows repository packages into the main repository.

Hint: if you use the MinGW compiler, don't forget to add to your PATH the path to libc dlls (usually C:\cygwin64\usr\x86_64-w64-mingw32\sys-root\mingw\bin). Or compile opam with make cold CONFIGURE_ARGS=--with-private-runtime, and if you change opam location, don't forget to copy Opam.Runtime.amd64 (or Opam.Runtime.i386) with it.

Recursive Pin

When installing or pinning a package using opam install or opam pin, opam normally only looks for opam files at the root of the installed package. With recursive pinning, you can now instruct opam to also look for .opam files in subdirectories, while maintaining the correct relationship between the .opam files and the package root for versioning and build purposes.

Recursive pinning is used with the following options to opam pin and opam install:

  • With --recursive, opam will look for .opam files recursively in all subdirectories.
  • With --subpath <path>, opam will only look for .opam files in the subdirectory <path>.

The two options can be combined: for instance, if your opam packages are stored as a deep hierarchy in the mylib subdirectory of your project, give opam pin . --recursive --subpath mylib a try!

You can use these options with opam pin, opam install, and opam remove.

$ tree .
.
├── ba
│   └── z
│       └── z.opam
├── bar
│   └── bar.opam
└── foo.opam

$ opam pin . --subpath ba/z --no-action
Package z does not exist, create as a NEW package? [y/n] y
z is now subpath-pinned to directory /ba/z in git+file:///tmp/recpin#master (version 0.1)

$ opam pin --recursive . --no-action
This will pin the following packages: foo, z, bar. Continue? [y/n] y
foo is now pinned to git+file:///tmp/recpin#master (version 0.1)
Package z does not exist, create as a NEW package? [y/n] y
z is now subpath-pinned to directory /ba/z in git+file:///tmp/recpin#master (version 0.1)
Package bar does not exist, create as a NEW package? [y/n] y
bar is now subpath-pinned to directory /bar in file:///tmp/recpin (version 0.1)

$ opam pin
bar.0.1  (uninstalled)  rsync  directory /bar in file:///tmp/recpin
foo.0.1  (uninstalled)  git    git+file:///tmp/recpin#master
z.0.1    (uninstalled)  git    directory /ba/z in git+file:///tmp/recpin#master

$ opam pin . --recursive --subpath ba/ --no-action
Package z does not exist, create as a NEW package? [y/n] y
z is now subpath-pinned to directory /ba/z in git+file:///tmp/recpin#master (version 0.1)

Tree View

opam tree shows packages and their dependencies with a tree view. It is very helpful to determine which packages bring which dependencies in your installed switch.

$ opam tree cppo
cppo.1.6.9
├── base-unix.base
├── dune.3.8.2 (>= 1.10)
│   ├── base-threads.base
│   ├── base-unix.base [*]
│   └── ocaml.4.14.1 (>= 4.08)
│       ├── ocaml-base-compiler.4.14.1 (>= 4.14.1~ & < 4.14.2~)
│       └── ocaml-config.2 (>= 2)
│           └── ocaml-base-compiler.4.14.1 (>= 4.12.0~) [*]
└── ocaml.4.14.1 (>= 4.02.3) [*]

It can also display a reverse-dependency tree (through opam why, which is an alias to opam tree --rev-deps). This is useful to examine how dependency versions get constrained.

$ opam why cmdliner
cmdliner.1.2.0
├── (>= 1.1.0) b0.0.0.5
│   └── (= 0.0.5) odig.0.0.9
├── (>= 1.1.0) ocp-browser.1.3.4
├── (>= 1.0.0) ocp-indent.1.8.1
│   └── (>= 1.4.2) ocp-index.1.3.4
│       └── (= version) ocp-browser.1.3.4 [*]
├── (>= 1.1.0) ocp-index.1.3.4 [*]
├── (>= 1.1.0) odig.0.0.9 [*]
├── (>= 1.0.0) odoc.2.2.0
│   └── (>= 2.0.0) odig.0.0.9 [*]
├── (>= 1.1.0) opam-client.2.2.0~alpha
│   ├── (= version) opam.2.2.0~alpha
│   └── (= version) opam-devel.2.2.0~alpha
├── (>= 1.1.0) opam-devel.2.2.0~alpha [*]
├── (>= 0.9.8) opam-installer.2.2.0~alpha
└── user-setup.0.7

Special thanks to @cannorin for contributing this feature.

Recommended Development Tools

There is now a way for a project maintainer to share their project development tools: the with-dev-setup dependency flag. It is used in the same way as with-doc and with-test: by adding a {with-dev-setup} filter after a dependency. It will be ignored when installing normally, but it's pulled in when the package is explicitely installed with the --with-dev-setup flag specified on the command line. The variable is also resolved in the post-messages: field to allow maintainers to share more informations about that setup.

This is typically useful for tools that are required for bootstrapping or regenerating artifacts.

For example

opam-version: "2.0"
depends: [
  "ocaml"
  "dune"
  "ocp-indent" {with-dev-setup}
]
build: [make]
install: [make "install"]
post-messages:
[ "Thanks for installing the package"
  "and its tool dependencies too, it will help for your futur PRs" {with-dev-setup} ]

Software Heritage Binding

Software Heritage is a project that aims to archive all software source code in existence. This is done by collecting source code with a loader that uploads software source code to the Software Heritage distributed infrastructure. From there, any project/version is available via the search webpage and via a unique identifier called the SWHID. Some OCaml source code is already archived, and the main opam and Coq repository packages are continuously uploaded.

Opam now integrates a fallback to Software Heritage archive retrieval, based on SWHID. If an SWHID URL is present in an opam file, the fallback can be activated.

To keep backwards compatibility of opam files, we added a specific Software Heritage URL syntax to the url.mirrors: field, which is used to specify mirrors of the main URL. Opam 2.2.+ understands this specific syntax as a Software Heritage fallback URL: https://swhid.opam.ocaml.org/<SWHID>.

url {
  src: "https://faili.ng/url.tar.gz"
  checksum: "sha512=e2146c1d7f53679fd22df66c9061b5ae4f8505b749513eedc67f3c304f297d92e54f5028f40fb5412d32c7d7db92592eacb183128d2b6b81d10ea716b7496eba"
  mirrors: [
    "https//failli.ng/mirror.tar.gz"
    "https://swhid.opam.ocaml.org/swh:1:dir:9f2be900491e1dabfc027848204ae01aa88fc71d"
  ]
}

To add a Software Heritage fallback URL to your package, use the swhid library. Specifically the Compute.directory_identifier_deep function:

  1. Download opam package archive
  2. Extract the archive
  3. Compute SWHID with Compute.directory_identifier_deep. You can use this oneliner in the directory:
ocaml -e '#use "topfind";; #require "digestif.ocaml";; #require "swhid";; Swhid_core.Object.pp Format.std_formatter (Result.get_ok (Swhid.Compute.directory_identifier_deep "."))'

Special thanks to @zapashcanon for collaborating on this feature.

Formula (Experimental)

It is now possible to leverage the full expressivity of package dependency formulas from the command line during switch creation and package operations.

It is possible to create a switch using a formula. For example, with ocaml-variant or ocaml-system, excluding ocaml-base-compiler:

opam switch create ocaml --formula '"ocaml-variants" {>= "4.14.1"} | "ocaml-system"'

This syntax is brought to install commands. For example, while installing a package, let's say genet, you can specify that you want to install either conf-mariadb & mariadb or conf-postgresql:

opam install genet --formula '["mysql" ("conf-mariadb" & "mariadb" | "conf-postgresql")]'

New Options

Here are several of new options (possibly scripts breaking changes are marked with ✘):

  • opam pin --current to fix a package to its current state (disabling pending reinstallations or removals from the repository). The installed package will be pinned with the opam file that is stored in opam internal state, the one that is currently installed.

  • opam pin remove --all to remove all the pinned packages from a switch.

  • opam pin remove pkg.version now removes the pins on pinned pkg.version.

  • opam exec --no-switch to remove opam environment from launched command.

$ export FOOVAR=env
$ opam show foo --field setenv
FOOVAR = "package"
$ opam exec  -- env | grep "OPAM_SWITCH\|FOO"
FOOVAR=package
OPAM_SWITCH_PREFIX=~/.opam/env
$ opam exec --no-switch -- env | grep "OPAM_SWITCH\|FOO"
FOOVAR=env

  • opam source --no-switch to allow downloading package sources without having an installed switch (instead of failing).

  • opam clean --untracked to remove untracked files interactively remaining from previous packages removal.

  • opam switch -, inspired from git switch -, that goes back to the previously selected global switch.

  • opam admin add-constraint <cst> --packages pkg1,pkg2,pkg3 to select a subset of packages to apply constraints.

  • ✘ Change --base into --invariant. opam switch compiler column now contains installed packages that verifies invariant formula, and empty synopsis shows switch invariant.

$ opam switch create inv --formula '["ocaml" {>= "4.14.1"} "dune"]'
$ opam switch invariant
["ocaml" {>= "4.14.1"} "dune"]
$ opam list --invariant
# Packages matching: invariant
# Name # Installed # Synopsis
dune   3.8.2       Fast, portable, and opinionated build system
ocaml  5.0.0       The OCaml compiler (virtual package)
$ opam switch list
#  switch   compiler                                            description
→  inv      ocaml-base-compiler.5.0.0,ocaml-options-vanilla.1   ocaml >= 4.14.1 & dune

Try It!

In case you plan a possible rollback, you may want to first backup your ~/.opam directory.

The upgrade instructions are unchanged:

  1. From binaries: run
bash -c "sh <(curl -fsSL https://raw.githubusercontent.com/ocaml/opam/master/shell/install.sh) --version 2.2.0~alpha"

Or download manually from the Github "Releases" page to your PATH.

  1. From source, manually: see the instructions in the README.

Then run:

opam init --reinit -ni

Please report any issues to the bug-tracker.

Thanks for trying this new release out, and we're hoping you will enjoy the new features!