OCaml Changelog

Bug fix release before major version.

We are pleased to announce a preview release for opam 2.0, with over 700 patches since 1.2.2. Version 2.0~alpha4 has just been released, and is ready to be more widely tested.

This version brings many new features and changes, the most notable one being that OCaml compiler packages are no longer special entities, and are replaced by standard package definition files. This in turn means that opam users have more flexibility in how switches are managed, including for managing non-OCaml environments such as Coq using the same familiar tools.

A Few Highlights

This is just a sample, see the full changelog for more:

• Sandboxed builds: Command wrappers can be configured to, for example, restrict permissions of the build and install processes using Linux namespaces, or run the builds within Docker containers.

• Compilers as packages: This brings many advantages for opam workflows, such as being able to upgrade the compiler in a given switch, better tooling for local compilers, and the possibility to define coq as a compiler or even use opam as a generic shell scripting engine with dependency tracking.

• Local switches: Create switches within your projects for easier management. Simply run opam switch create <directory> <compiler> to get started.

• Inplace build: Use opam to build directly from your source directory. Ensure the package is pinned locally then run opam install --inplace-build.

• Automatic file tracking:: opam now tracks the files installed by packages and is able to cleanly remove them when no existing files were modified. The remove: field is now optional as a result.

• Configuration file: This can be used to direct choices at opam init automatically (e.g. specific repositories, wrappers, variables, fetch commands, or the external solver). This can be used to override all of opam's OCaml-related settings.

• Simpler library: the OCaml API is completely rewritten and should make it much easier to write external tools and plugins. Existing tools will need to be ported.

• Better error mitigation: Through clever ordering of the shell actions and separation of build and install, most build failures can keep your current installation intact, not resulting in removed packages anymore.

Roll out

You are very welcome to try out the alpha, and report any issues. The repository at opam.ocaml.org will remain in 1.2 format (with a 2.0 mirror at opam.ocaml.org/2.0~dev in sync) until after the release is out, which means the extensions can not be used there yet, but you are welcome to test on local or custom repositories, or package pinnings. The reverse translation (2.0 to 1.2) is planned, to keep supporting 1.2 installations after that date.

The documentation for the new version is available at https://opam.ocaml.org/doc/2.0/. This is still work in progress, so please do ask if anything is unclear.

Interface changes

Commands opam switch and opam list have been rehauled for more consistency and flexibility: the former won't implicitly create new switches unless called with the create subcommand, and opam list now allows to combine filters and finely specify the output format. They may not be fully backwards compatible, so please check your scripts.

Most other commands have also seen fixes or improvements. For example, opam doesn't forget about your set of installed packages on the first error, and the new opam install --restore can be used to reinstall your selection after a failed upgrade.

Repository changes

While users of opam 1.2 should feel at home with the changes, the 2.0 repository and package formats are not compatible. Indeed, the move of the compilers to standard packages implies some conversions, and updates to the relationships between packages and their compiler. For example, package constraints like

available: [ ocaml-version >= "4.02" ]

are now written as normal package dependencies:

depends: [ "ocaml" {>= "4.02"} ]

To make the transition easier,

• upgrade of a custom repository is simply a matter of running opam-admin upgrade-format at its root;
• the official repository at opam.ocaml.org already has a 2.0 mirror, to which you will be automatically redirected;
• packages definition are automatically converted when you pin a package.

Note that the ocaml package on the official repository is actually a wrapper that depends on one of ocaml-base-compiler, ocaml-system or ocaml-variants, which contain the different flavours of the actual compiler. It is expected that it may only get picked up when requested by package dependencies.

Package format changes

The opam package definition format is very similar to before, but there are quite a few extensions and some changes:

• it is now mandatory to separate the build: and install: steps (this allows tracking of installed files, better error recovery, and some optional security features);
• the url and description can now optionally be included in the opam file using the section url {} and fields synopsis: and description:;
• it is now possible to have dependencies toggled by globally-defined opam variables (e.g. for a dependency needed on some OS only), or even rely on the package information (e.g. have a dependency at the same version);
• the new setenv: field allows packages to export updates to environment variables;
• custom fields x-foo: can be used for extensions and external tools;
• allow """ delimiters around unescaped strings
• & is now parsed with higher priority than |
• field ocaml-version: can no longer be used
• the remove: field should not be used anymore for simple cases (just removing files)

Let's go then -- how to try it ?

First, be aware that you'll be prompted to update your ~/.opam to 2.0 format before anything else, so if you value it, make a backup. Or just export OPAMROOT to test the alpha on a temporary opam root.

Packages for opam 2.0 are already in the opam repository, so if you have a working opam installation of opam (at least 1.2.1), you can bootstrap as easily as:

opam install opam-devel

This doesn't install the new opam to your PATH within the current opam root for obvious reasons, so you can manually install it as e.g. "opam2" using:

sudo cp $(opam config var "opam-devel:lib")/opam /usr/local/bin/opam2

You can otherwise install as usual:

• Using pre-built binaries (available for OSX and Linux x86, x86_64, armhf) and our install script:

  wget https://raw.github.com/ocaml/opam/2.0-alpha4-devel/shell/opam_installer.sh -O - | sh -s /usr/local/bin

  Equivalently, pick your version and download it to your PATH;

• Building from our inclusive source tarball: download here and build using ./configure && make lib-ext && make && make install if you have OCaml >= 4.01 already available, make cold && make install otherwise;

• Or from source, following the included instructions from the README. Some files have been moved around, so if your build fails after you updated an existing git clone, try to clean it up (git clean -fdx).

This release also contains contributions from: Rudi Grinberg, Fourchaux, Christopher Reichert, David Allsopp, Nick Borden, Mario Rodas, @Twinside, Pierre Chambart, Philipp Haselwarter, Tomasz Kołodziejski and Syohei Yoshida.

OPAM 1.2.2 has just been released. This fixes a few issues over 1.2.1 and brings a couple of improvements, in particular better use of the solver to keep the installation as up-to-date as possible even when the latest version of a package can not be installed.

Upgrade from 1.2.1 (or earlier)

See the normal installation instructions: you should generally pick up the packages from the same origin as you did for the last version -- possibly switching from the official repository packages to the ones we provide for your distribution, in case the former are lagging behind.

There are no changes in repository format, and you can roll back to earlier versions in the 1.2 branch if needed.

Improvements

• Conflict messages now report the original version constraints without translation, and they have been made more concise in some cases
• Some new opam lint checks, opam lint now numbers its warnings and may provide script-friendly output
• Feature to automatically install plugins, e.g. opam depext will prompt to install depext if available and not already installed
• Priority to newer versions even when the latest can't be installed (with a recent solver only. Before, all non-latest versions were equivalent to the solver)
• Added opam list --resolve to list a consistent installation scenario
• Be cool by default on errors in OPAM files, these don't concern end-users and packagers and CI now have opam lint to check them.

Fixes

• OSX: state cache got broken in 1.2.1, which could induce longer startup times. This is now fixed
• opam config report has been fixed to report the external solver properly
• --dry-run --verbose properly outputs all commands that would be run again
• Providing a simple path to an aspcud executable as external solver (through options or environment) works again, for backwards-compatibility
• Fixed a fd leak on solver calls (thanks Ivan Gotovchits)
• opam list now returns 0 when no packages match but no pattern was supplied, which is more helpful in scripts relying on it to check dependencies.

OPAM 1.2.1 has just been released. This patch version brings a number of fixes and improvements over 1.2.0, without breaking compatibility.

Upgrade from 1.2.0 (or earlier)

See the normal installation instructions: you should generally pick up the packages from the same origin as you did for the last version -- possibly switching from the official repository packages to the ones we provide for your distribution, in case the former are lagging behind.

What's new

No huge new features in this point release -- which means you can roll back to 1.2.0 in case of problems -- but lots going on under the hood, and quite a few visible changes nonetheless:

• The engine that processes package builds and other commands in parallel has been rewritten. You'll notice the cool new display but it's also much more reliable and efficient. Make sure to set jobs: to a value greater than 1 in ~/.opam/config in case you updated from an older version.
• The install/upgrade/downgrade/remove/reinstall actions are also processed in a better way: the consequences of a failed actions are minimised, when it used to abort the full command.
• When using version control to pin a package to a local directory without specifying a branch, only the tracked files are used by OPAM, but their changes don't need to be checked in. This was found to be the most convenient compromise.
• Sources used for several OPAM packages may use <name>.opam files for package pinning. URLs of the form git+ssh:// or hg+https:// are now allowed.
• opam lint has been vastly improved.

... and much more

There is also a new manual documenting the file and repository formats.

Fixes

See the changelog for a summary or closed issues in the bug-tracker for an overview.

Experimental features

These are mostly improvements to the file formats. You are welcome to use them, but they won't be accepted into the official repository until the next release.

• New field features: in opam files, to help with ./configure scripts and documenting the specific features enabled in a given build. See the original proposal and the section in the new manual
• The "filter" language in opam files is now well defined, and documented in the manual. In particular, undefined variables are consistently handled, as well as conversions between string and boolean values, with new syntax for converting bools to strings.
• New package flag "verbose" in opam files, that outputs the package's build script to stdout
• New field libexec: in <name>.install files, to install into the package's lib dir with the execution bit set.
• Compilers can now be defined without source nor build instructions, and the base packages defined in the packages: field are now resolved and then locked. In practice, this means that repository maintainers can move the compiler itself to a package, giving a lot more flexibility.

Main new feature is a faster short-path, and also a lot of buxfixes.

This release also contains contributions from: Geoff Gole, Rudi Grinberg, Markus Mottl, Roman Vorobets and Arthur Wendling.

We are very proud to announce the availability of OPAM 1.2.0.

Upgrade from 1.1

Simply follow the usual instructions, using your preferred method (package from your distribution, binary, source, etc.) as documented on the homepage.

NOTE: There are small changes to the internal repository format (~/.opam). It will be transparently updated on first run, but in case you might want to go back and have anything precious there, you're advised to back it up.

Usability

Lot of work has been put into providing a cleaner interface, with helpful behaviour and messages in case of errors.

The documentation pages also have been largely rewritten for consistency and clarity.

New features

This is just the top of the list:

• A extended and versatile opam pin command. See the Simplified packaging workflow
• More expressive queries, see for example opam source
• New metadata fields, including source repositories, bug-trackers, and finer control of package behaviour
• An opam lint command to check the quality of packages

For more detail, see the announcement for the beta, the full changelog, and the bug-tracker.

Package format

The package format has been extended to the benefit of both packagers and users. The repository already accepts packages in the 1.2 format, and this won't affect 1.1 users as a rewrite is done on the server for compatibility with 1.1.

If you are hosting a repository, you may be interested in these administration scripts to quickly take advantage of the new features or retain compatibility.