OLD | NEW |
| (Empty) |
1 --- | |
2 title: "Versioning Philosophy" | |
3 --- | |
4 | |
5 1. [A name and a number](#a-name-and-a-number) | |
6 1. [Shared dependencies and unshared libraries](#shared-dependencies-and-unshare
d-libraries) | |
7 1. [Version lock](#version-lock) | |
8 1. [Version constraints](#version-constraints) | |
9 1. [Semantic versions](#semantic-versions) | |
10 1. [Constraint solving](#constraint-solving) | |
11 1. [Constraint context](#constraint-context) | |
12 1. [Lockfiles](#lockfiles) | |
13 1. [When things go wrong](#when-things-go-wrong) | |
14 1. [Summary](#summary) | |
15 {:.toc} | |
16 | |
17 One of pub's main jobs is helping you work with versioning. Here, I'll | |
18 explain a bit about the history of versioning and pub's approach to it. | |
19 Consider this to be advanced information. If you want a better picture of *why* | |
20 pub was designed the way it was, read on. If you just want to *use* pub, the | |
21 [other docs](index.html) will serve you better. | |
22 | |
23 Modern software development, especially web development, leans heavily on | |
24 reusing lots and lots of existing code. That includes code *you* wrote in the | |
25 past, but also stuff from third-parties, everything from big frameworks to tiny | |
26 little utility libraries. It's not uncommon for an application to depend on | |
27 dozens of different packages and libraries. | |
28 | |
29 It's hard to understate how awesome this is. When you see stories of tiny web | |
30 startups building a site in a few weeks that gets millions of users, the | |
31 only reason they can pull that off is because the open source community has | |
32 laid a feast of software at their feet. | |
33 | |
34 But there's still no such thing as a free lunch. There's a challenge to code | |
35 reuse, especially reusing code you don't maintain. When your app uses tons of | |
36 code being developed by other people, what happens when they change it? They | |
37 don't want to break your app, and you certainly don't either. | |
38 | |
39 ## A name and a number | |
40 | |
41 We solve this by *versioning*. When you depend on some piece of outside code, | |
42 you don't just say "My app uses `widgets`." You say, "My app uses | |
43 `widgets 2.0.5`." That combination of name and version number uniquely | |
44 identifies an *immutable* chunk of code. The people hacking on `widgets` can | |
45 make all of the changes they want, but they promise to not touch any already | |
46 released versions. They can put out `2.0.6` or `3.0.0` and it won't affect you | |
47 one whit because the version you use is unchanged. | |
48 | |
49 When you *do* want to get those changes, you can always point your app to a | |
50 newer version of `widgets` and you don't have to coordinate with those | |
51 developers to do it. So, problem solved, right? | |
52 | |
53 ## Shared dependencies and unshared libraries | |
54 | |
55 Well, no. Depending on specific versions works fine when your dependency *graph* | |
56 is really just a dependency *tree*. If your app depends on a bunch of stuff, and | |
57 those things in turn have their own dependencies and so on, that all works fine | |
58 as long as none of those dependencies *overlap*. | |
59 | |
60 But let's consider an example: | |
61 | |
62 myapp | |
63 / \ | |
64 / \ | |
65 widgets templates | |
66 \ / | |
67 \ / | |
68 collections | |
69 | |
70 So your app uses `widgets` and `templates`, and *both* of those use | |
71 `collections`. This is called a **shared dependency**. Now what happens when | |
72 `widgets` wants to use `collections 2.3.5` and `templates` wants | |
73 `collections 2.3.7`? What if they don't agree on a version? | |
74 | |
75 One option is to just let the app use both | |
76 versions of `collections`. It will have two copies of the library at different | |
77 versions and `widgets` and `templates` will each get the one they want. | |
78 | |
79 This is what [npm][] does for node.js. Would it work for Dart? Consider this | |
80 scenario: | |
81 | |
82 1. `collections` defines some `Dictionary` class. | |
83 2. `widgets` gets an instance of it from its copy of `collections` (`2.3.5`). | |
84 It then passes it up to `myapp`. | |
85 3. `myapp` sends the dictionary over to `templates`. | |
86 4. That in turn sends it down to *its* version of `collections` (`2.3.7`). | |
87 5. The method that takes it has a `Dictionary` type annotation for that object. | |
88 | |
89 As far as Dart is concerned, `collections 2.3.5` and `collections 2.3.7` are | |
90 entirely unrelated libraries. If you take an instance of class `Dictionary` from | |
91 one and pass it to a method in the other, that's a completely different | |
92 `Dictionary` type. That means it will fail to match a `Dictionary` type | |
93 annotation in the receiving library. Oops. | |
94 | |
95 Because of this (and because of the headaches of trying to debug an app that | |
96 has multiple versions of things with the same name), we've decided npm's model | |
97 isn't a good fit. | |
98 | |
99 [npm]: https://npmjs.org/ | |
100 | |
101 ## Version lock | |
102 | |
103 Instead, when you depend on a package, your app will only use a single copy of | |
104 that package. When you have a shared dependency, everything that depends on it | |
105 has to agree on which version to use. If they don't, you get an error. | |
106 | |
107 That doesn't actually solve your problem though. When you *do* get that error, | |
108 you need to be able to resolve it. So let's say you've gotten yourself into | |
109 that situation in the above example. You want to use `widgets` and `templates`, | |
110 but they are using different versions of `collections`. What do you do? | |
111 | |
112 The answer is to try to upgrade one of those. `templates` wants | |
113 `collections 2.3.7`. Is there a later version of `widgets` that you can upgrade | |
114 to that works with that version? | |
115 | |
116 In many cases, the answer will be "no". Look at it from the perspective of the | |
117 people developing `widgets`. They want to put out a new version with new changes | |
118 to *their* code, and they want as many people to be able to upgrade to it it as | |
119 possible. If they stick to their *current* version of `collections` then anyone | |
120 who is using the current version `widgets` will be able to drop in this new one | |
121 too. | |
122 | |
123 If they were to upgrade *their* dependency on `collections` then everyone who | |
124 upgrades `widgets` would have to as well, *whether they want to or not.* That's | |
125 painful, so you end up with a disincentive to upgrade dependencies. That's | |
126 called **version lock**: everyone wants to move their dependencies forward, but | |
127 no one can take the first step because it forces everyone else to as well. | |
128 | |
129 ## Version constraints | |
130 | |
131 To solve version lock, we loosen the constraints that packages place on their | |
132 dependencies. If `widgets` and `templates` can both indicate a *range* of | |
133 versions for `collections` that they will work with, then that gives us enough | |
134 wiggle room to move our dependencies forward to newer versions. As long as there | |
135 is overlap in their ranges, we can still find a single version that makes them | |
136 both happy. | |
137 | |
138 This is the model that [bundler](http://gembundler.com/) follows, and is pub's | |
139 model too. When you add a dependency in your pubspec, you can specify a *range* | |
140 of versions that you can accept. If the pubspec for `widgets` looked like this: | |
141 | |
142 {% highlight yaml %} | |
143 dependencies: | |
144 collections: '>=2.3.5 <2.4.0' | |
145 {% endhighlight %} | |
146 | |
147 Then we could pick version `2.3.7` for `collections` and then both `widgets` | |
148 and `templates` have their constraints satisfied by a single concrete version. | |
149 | |
150 ## Semantic versions | |
151 | |
152 When you add a dependency to your package, you'll sometimes want to specify a | |
153 range of versions to allow. How do you know what range to pick? You need to | |
154 forward compatible, so ideally the range encompasses future versions that | |
155 haven't been released yet. But how do you know your package is going to work | |
156 with some new version that doesn't even exist yet? | |
157 | |
158 To solve that, you need to agree on what a version number *means*. Imagine that | |
159 the developers of a package you depend on say, "If we make any backwards | |
160 incompatible change, then we promise to increment the major version number." | |
161 If you trust them, then if you know your package works with `2.5.7` of theirs, | |
162 you can rely on it working all the way up to `3.0.0`. So you can set your range | |
163 like: | |
164 | |
165 {% highlight yaml %} | |
166 dependencies: | |
167 collections: '>=2.3.5 <3.0.0' | |
168 {% endhighlight %} | |
169 | |
170 To make this work, then, we need to come up with that set of promises. | |
171 Fortunately, other smart people have done the work of figuring this all out and | |
172 named it [*semantic versioning*](http://semver.org/). | |
173 | |
174 That describes the format of a version number, and the exact API behavioral | |
175 differences when you increment to a later version number. Pub requires versions | |
176 to be formatted that way, and to play well with the pub community, your package | |
177 should follow the semantics it specifies. You should assume that the packages | |
178 you depend on also follow it. (And if you find out they don't, let their | |
179 authors know!) | |
180 | |
181 We've got almost all of the pieces we need to deal with versioning and API | |
182 evolution now. Let's see how they play together and what pub does. | |
183 | |
184 ## Constraint solving | |
185 | |
186 When you define your package, you list its | |
187 [**immediate dependencies**](glossary.html#immediate-dependency)—the | |
188 packages it itself uses. For each one, you specify the range of versions it | |
189 allows. Each of those dependent packages may in turn have their own | |
190 dependencies (called | |
191 [**transitive dependencies**](glossary.html#transitive-dependency). Pub will | |
192 traverse these and build up the entire deep dependency graph for your app. | |
193 | |
194 For each package in the graph, pub looks at everything that depends on it. It | |
195 gathers together all of their version constraints and tries to simultaneously | |
196 solve them. (Basically, it intersects their ranges.) Then it looks at the | |
197 actual versions that have been released for that package and selects the best | |
198 (most recent) one that meets all of those constraints. | |
199 | |
200 For example, let's say our dependency graph contains `collections`, and three | |
201 packages depend on it. Their version constraints are: | |
202 | |
203 >=1.7.0 | |
204 >=1.4.0 <2.0.0 | |
205 <1.9.0 | |
206 | |
207 The developers of `collections` have released these versions of it: | |
208 | |
209 1.7.0 | |
210 1.7.1 | |
211 1.8.0 | |
212 1.8.1 | |
213 1.8.2 | |
214 1.9.0 | |
215 | |
216 The highest version number that fits in all of those ranges is `1.8.2`, so pub | |
217 picks that. That means your app *and every package your app uses* will all use | |
218 `collections 1.8.2`. | |
219 | |
220 ## Constraint context | |
221 | |
222 The fact that selecting a package version takes into account *every* package | |
223 that depends on it has an important consequence: *the specific version that | |
224 will be selected for a package is a global property of the app using that | |
225 package.* | |
226 | |
227 I'll walk through an example so you can see what this means. Let's say we have | |
228 two apps. Here are their pubspecs: | |
229 | |
230 {% highlight yaml %} | |
231 name: my_app | |
232 dependencies: | |
233 widgets: | |
234 {% endhighlight %} | |
235 | |
236 {% highlight yaml %} | |
237 name: other_app | |
238 dependencies: | |
239 widgets: | |
240 collections: '<1.5.0' | |
241 {% endhighlight %} | |
242 | |
243 They both depend on `widgets`, whose pubspec is: | |
244 | |
245 {% highlight yaml %} | |
246 name: widgets | |
247 dependencies: | |
248 collections: '>=1.0.0 <2.0.0' | |
249 {% endhighlight %} | |
250 | |
251 The `other_app` package uses depends directly on `collections` itself. The | |
252 interesting part is that it happens to have a different version constraint on | |
253 it than `widgets` does. | |
254 | |
255 What this means is that you can't just look at the `widgets` package in | |
256 isolation to figure out what version of `collections` it will use. It depends | |
257 on the context. In `my_app`, `widgets` will be using `collections 1.9.9`. But | |
258 in `other_app`, `widgets` will get saddled with `collections 1.4.9` because of | |
259 the *other* constraint that `otherapp` places on it. | |
260 | |
261 This is why each app gets its own "packages" directory: The concrete version | |
262 selected for each package depends on the entire dependency graph of the | |
263 containing app. | |
264 | |
265 ## Lockfiles | |
266 | |
267 So once pub has solved your app's version constraints, then what? The end | |
268 result is a complete list of every package that your app depends on either | |
269 directly or indirectly and the best version of that package that will work with | |
270 your app's constraints. | |
271 | |
272 Pub takes that and writes it out to a **lockfile** in your app's directory | |
273 called `pubspec.lock`. When pub builds the "packages" directory your app, it | |
274 uses the lockfile to know what versions of each package to pull in. (And if | |
275 you're curious to see what versions it selected, you can read the lockfile to | |
276 find out.) | |
277 | |
278 The next important thing pub does is it *stops touching the lockfile*. Once | |
279 you've got a lockfile for your app, pub won't mess with it until you tell it to. | |
280 This is important. It means you won't spontanteously start using new versions | |
281 of random packages in your app without intending to. Once your app is locked, | |
282 it stays locked until you manually tell it to update the lockfile. | |
283 | |
284 If your package is for an app, you take your lockfile *check that bad boy | |
285 into your source control system!* That way, everyone on your team will be using | |
286 the exact same versions of every dependency when they hack on your app. You'll | |
287 also use this when you deploy your app so you can ensure that your production | |
288 servers are using the exact same packages that you're developing with. | |
289 | |
290 ## When things go wrong | |
291 | |
292 Of course, all of this presumes that your dependency graph is perfect and | |
293 flawless. Oh, to be so fortunate. Even with version ranges and pub's constraint | |
294 solving and semantic versioning, you can never be entirely spared from the | |
295 dangers of version hell. | |
296 | |
297 There are a couple of problems you can run into: | |
298 | |
299 ### You can have disjoint constraints | |
300 | |
301 Lets say your app uses `widgets` and | |
302 `templates` and both use `collections`. But `widgets` asks for a version | |
303 of it between `1.0.0` and `2.0.0` and `templates` wants something | |
304 between `3.0.0` and `4.0.0`. Those ranges don't even overlap. There's no | |
305 possible version that would work. | |
306 | |
307 ### You can have ranges that don't contain a released version | |
308 | |
309 Let's say after | |
310 putting all of the constraints on a shared dependency together, you're | |
311 left with the narrow range of `>=1.2.4 <1.2.6`. It's not an empty range. | |
312 If there was a version `1.2.4` of the dependency, you'd be golden. But maybe | |
313 they never released that and instead when straight from `1.2.3` to `1.3.0`. | |
314 You've got a range but nothing exists inside it. | |
315 | |
316 ### You can have an unstable graph | |
317 | |
318 This is, by far, the hairiest part of | |
319 pub's version solving process. I've described the process as "build up the | |
320 dependency graph and then solve all of the constraints and pick versions". | |
321 But it doesn't actually work that way. How could you build up the *whole* | |
322 dependency graph before you've picked *any* versions? *The pubspecs | |
323 themselves are version-specific*. Different versions of the same package | |
324 may have different sets of dependencies. | |
325 | |
326 As you're selecting versions of packages, they are changing the shape of | |
327 the dependency graph itself. As the graph changes, that may change | |
328 constraints, which can cause you to select different versions, and then you | |
329 go right back around in a circle. | |
330 | |
331 Sometimes this process never settles down into a stable solution. Gaze into | |
332 the abyss: | |
333 | |
334 {% highlight yaml %} | |
335 name: my_app | |
336 version: 0.0.0 | |
337 dependencies: | |
338 yin: '>=1.0.0' | |
339 {% endhighlight %} | |
340 | |
341 {% highlight yaml %} | |
342 name: yin | |
343 version: 1.0.0 | |
344 dependencies: | |
345 {% endhighlight %} | |
346 | |
347 {% highlight yaml %} | |
348 name: yin | |
349 version: 2.0.0 | |
350 dependencies: | |
351 yang: '1.0.0' | |
352 {% endhighlight %} | |
353 | |
354 {% highlight yaml %} | |
355 name: yang | |
356 version: 1.0.0 | |
357 dependencies: | |
358 yin: '1.0.0' | |
359 {% endhighlight %} | |
360 | |
361 In all of these cases, there is no set of concrete versions that will work for | |
362 your app, and when this happens pub will report an error and tell you what's | |
363 going on. It definitely will not try to leave you in some weird state where you | |
364 think things can work but won't. | |
365 | |
366 ## Summary | |
367 | |
368 Wow, that's a lot to get through. Here's the important bits: | |
369 | |
370 * Code reuse is great, but in order to let developers move quickly, packages | |
371 need to be able to evolve independently. | |
372 * Versioning is how you enable that. But depending on single concrete versions | |
373 is too precise and with shared dependencies leads to version lock. | |
374 * To cope with that, you depend on *ranges* of versions. Pub will then walk | |
375 your dependency graph and pick the best versions for you. If it can't, it | |
376 tells you. | |
377 * Once your app has a solid set of versions for its dependencies, that gets | |
378 pinned down in a *lockfile*. That ensures that every machine your app is | |
379 on is using the same versions of all of its dependencies. | |
OLD | NEW |