> Version numbers are are notoriously unreliable in predicting if something breaks after [a dependency's] upgrade.
> [W]hen deciding if a given version of a given dependency is compatible with a dependant piece of software, instead of relying on version numbers rely on tests that actually verify the functionality and behaviour that piece of software actually depends on.
@rysiek @tomasino Looks like an interesting idea, but it seems to me this seems to assume fully static compilation and no dynamic libraries (e.g. this is common in Rust, and some consider it a drawback).
How would this work for platforms which prefer dynamic linking?
Would each binary include the compiled test-suite for all direct dependencies and run the tests at load time? This might have scalability problems on binary sizes and loading times.
I guess one thing is that it could be a question of dependency-management during development (say, managing your project's requirements.txt or package.json, or Gemfile), vs. user-facing dependency management (i.e. package managers on Linux distros, etc).
Perhaps one sane-ish solution would be for developers to do CBDM, and then "pin" versions numbers for user-facing dependency management?
I would prefer something more though: 1/
That way CBDM could be done even for user-facing dependency management on, say, their Linux or BSD distro, without the cost of running those tests for all dependencies when a package gets updated.
Of course, they would have an option to run them (just as they have an option to build those packages from source). 2/end
For right now I think CBDM will have to be opportunistic and preferential. If it's available it provides a better mechanism than semver, but it's not going to be everywhere at once. I'd like to see language specific implementations in development first. If the metadata for those can be abstracted or summarized into build metadata then we could potentially see CBDM in use in user-space as well. Generate summarized test coverage is still superior to a SemVer on its own.
Eventually if toolchains can communicate this sort of coverage I'd hope to see programming tooling automatically detect your usage of libraries and generate the dependency list automatically. "You're using X, it has been added to the test contract". It could also warn on-the-fly that you're adding a feature that's not covered. Most of this work I envision happening via the static test coverage summaries (whatever those look like in practice).
That future CBDM will also want to allow for integration testing mechanisms as part of your contracts. It may not be enough to know that LibBee passes unit tests for a certain feature. I may also need it to plug into my code in just the right way.
Regardless of future dreaming, I think there's enough here already to play with and test out in an individual development language environment. I hope these posts and discussion encourage some folk to try it.
Sure, and I also appreciated the answers by @rysiek .
Actually rereading my original answer to your toot, I realize it might appear more hostile than I intended.
Yes, I see risks and potential pitfalls, and I am not sure how the solutions would look like, but I do like the conceptual approach of CBDM and, as you said, it is worth discussing and experimenting with alternatives to the current models!
The difference I think is that autotools was focused on what the developers can do within their own project; meanwhile CBDM is broader and focuses more on the ecosystem -- write your tests, submit them upstream, and use them for dependency management.
But a very valid point, I need to dive into autotools! Thanks!
@bremner @romen @rysiek the sentiment is certainly behind it. A lot of my inspiration came from webdev where we learned the hard way to test for feature support rather than use browser detection. Transferring that idea to libraries the question becomes, "how can I know that this feature is supported?" The answer that came to me was to use tests as that contract.
If you've got deeper knowledge of other methodologies or ways to answer the question, or of other platforms that have explored this for dependency management please jump in the discussion as much as possible.
This Mastodon instance is for people interested in technology. Discussions aren't limited to technology, because tech folks shouldn't be limited to technology either!