Why do we keep building rotten foundations?

APIs are like bones: sometimes you have to break them to mend them, and it fucking hurts.

It’s not a perfect analogy, but it reflects an observation of (superficially) pragmatic behavioural tendencies, even if not a necessary truth. A broken implementation is bad, but it can be repaired. A broken API can be repaired too, but anything built on it then also needs to be fixed. And often when this occurs we like to throw the baby away with the bath water, start afresh, make a new clean design where we apply the lessons learned in the past attempts and produce something that, this time, will be perfect. Except, of course, we don’t always remember all the lessons, and it never is.

For instance, look at GTK. There was a recent blog post detailing plans for GTK development (and a further followup), which starts by discussing:

… the desire to create a modern toolkit with new features vs. the need to keep a stable API.

… which in my opinion, should not be a conflict, but let’s work our way there. Some choice snippets:

… and created a hesitation to expose “too much” API for fear of having to live with it “forever”.

(A legitimate concern, and one that gets to the heart of the matter, but let’s stick to the piece for now).

We want to improve this, and we have a plan.

And of course I read this and I am worried.

“Don’t worry Mr B, I have a cunning plan to solve the problem”

 We are going to increase the speed at which we do releases of new major versions of Gtk (ie: Gtk 4, Gtk 5, Gtk 6…). We want to target a new major release every two years.


The new release of Gtk is going to be fully parallel-installable with the old one. Gtk 4 and Gtk 3 will install alongside each other in exactly the same way as Gtk 2 and Gtk 3 — separate library name, separate pkg-config name, separate header directory. You will be able to have a system that has development headers and libraries installed for each of Gtk 2, 3, 4 and 5, if you want to do that.

No! DO NOT WANT! It’s bad enough having Gtk 2 and Gtk 3 (thankfully Gtk 1 is long gone), and you want me to have to litter my system with even more Gtk-sized turds… Please, no.

Oh well, I guess it can’t get any wors-

Meanwhile, Gtk 4.0 will not be the final stable API of what we would call “Gtk 4”. Each 6 months, the new release (Gtk 4.2, Gtk 4.4, Gtk 4.6) will break API and ABI vs. the release that came before it.

Oh for fucks sake, really? So your “major new release every two years” is, in effect, actually a major new pain-in-the-proverbial every 6 months fuck fuck fuck.

We will, of course, bump the soname with each new incompatible release — you will be able to run Gtk 4.0 apps alongside Gtk 4.2 and 4.4 apps, but you won’t be able to build them on the same system

Right, so every 6 months there will effectively be a new version of GTK with its own set of libraries and its own themes (getting to that, bear with me) and our systems will become a myriad of GTK because all the new apps developed during this time aren’t going to have any clue about which version they should target and trying to keep up the with latest API is going to be a futile effort because it’ll pretty much keep changing out from under your feet. Shit, there are still apps making the transition from Gtk 2 to Gtk 3 and they’ve literally had years. (I could at this point tell you how I much prefer Gtk 2 apps anyway, but there’s probably enough material there for a whole other blog post).

Before each new “dot 0” release, the last minor release on the previous major version will be designated as this “API stable” release. For Gtk 4, for example, we will aim for this to be 4.6 (and so on for future major releases).

I could mention that this seems like exactly the opposite of how major versions should work but, oops I just did. Just to illustrate:

4.0: development, 4.1: development, 4.2: development, …, 4.6: stable, 5.0: development, …

I mean, if 4.6 is the culmination of a series of changes leading to the next major release, then why isn’t it the next major release? Since when does “X.0” not designate a stable release?

“Gtk 4.0” is the first raw version of what will eventually grow into “Gtk 4”, sometime around Gtk 4.6

Oh lordy just listen to yourself for one second.

But really, forget about the ridiculous version numbering scheme; the real problem is the rapid-fire development with lack of API/ABI stability producing a plethora of incompatible versions. Why is this even necessary? Could I suggest that, rather than pumping out new and exciting APIs/features at a faster pace, what GTK really needs to do is sit down and flesh out a decent API that it won’t have to break every 6 months, for crying out loud. Because people are going to try to build software on top of your crappy toolkit, and you should accept some responsibility for the API design rather than kicking them in the nether regions twice a year. And if you really don’t want people to try and keep up with the latest and greatest and expect them to keep developing on GTK 3 despite that fact that you’re up to version 4.4 (or whatever) then at least use a sensible versioning scheme that reflects the notion of development and instability without having to learn that GTK devs used a different scheme than the rest of the software world, just because they felt like it.

this gives many application authors and desktop environments something that they have been asking for for a long time: a version of Gtk that has the features of Gtk 3, but the stability of Gtk 2.

You could give them that right now if you would just stop screwing around with the API. An API is stable if you don’t mess with it. And if you’re having to mess with your API on a continual basis, you’re doing it wrong. Like, the whole software development thing, all wrong.

By all means, declare parts of your API as unstable, document them as such and adjust them during a period of development, and then declare them stable (and never touch them again!). But the notion that you think it’s OK to potentially break the whole thing (or more accurately, any part of the whole thing) at any moment is disturbing. (Maybe that’s not what’s really meant; I certainly hope so, but it’s not clear). This idea that you’ll bang away in a frenzy with your software hammers for a few years and what comes out at the end will be “stable” is just hokey. It’s the wrong approach. You’re off the runway and into the harbour.

I really wonder, would it have been so hard to have GTK 3 add to the GTK 2 API rather than actually break it? I mean I know there’s the whole CSS thing (which still feels like a sick joke, frankly, especially because of the half-arsed implementation) and HiDPI support, but could these not have been implemented on top of the existing API? Could not the GDK API have been implemented on top of Cairo (if it wasn’t already), and retained for backwards compatibility? Etc etc? Yes, it’d be more work – much more work, perhaps – but that work only needs to get done once, rather than in each separate application that builds on the toolkit. And if it’d had been thought through properly at the start, it could have given us a new toolkit with a lot less pain.

The Wikipedia entry for GTK claims:

The most common criticism towards GTK+ is a lack of backwards-compatibility in major updates, most notably in the API[21] and theming.[22]

(I promise I did not edit that in myself. And I mentioned theming earlier, so I should elaborate: not only are GTK 2 themes not compatible with GTK 3, but GTK 3 themes aren’t either… it seems GTK 3.18 themes don’t generally work well with GTK 3.20, for instance).

That it’s so generally acknowledged that the API stability sucks is really telling.

GTK is a foundation library. And it’s rotten, and they know its rotten. And they keep tearing it down and replacing it – with another rotten foundation. And we all suffer because of it.

Note: I started out writing this post intending to discuss API breakage in general; unfortunately GTK is such an easy target that the whole post became about the one toolkit.

29 thoughts on “Why do we keep building rotten foundations?

  1. Well, in the words of my two favorite software dev related quotes:

    ‘How could the wise man build his house on the sand? How could the wise man build his house, where there is no foundation?’ — Eek a Mouse, Noah’s Ark

    ‘The only way a wise man would build his house on the sand is if it was just a hut and he was really high and really enjoyed building new huts.’ — random youtube comment on the above song (sadly I forget who the sage was that wrote that)

  2. The author’s tone of indignant outrage leaves me nonplussed. There are people who want to build open source, then there are those who want to tear it down… I can understand disagreements between developers with skin (i.e. code) in the game, but when onlookers tear apart projects that they’re not involved in, it just seems awfully mean-spirited.

    1. What kind of partisan nonsense is this? This isn’t the nineties, we’re not fighting board wars on Slashdot and spelling Microsoft with a $. These are perfectly valid criticisms and it has nothing to do with it being open-source. You can talk about ‘skin in the game’ all you like, but if I saw this and was considering leveraging GTK, this (their plan, not the article) would be a direct incentive to not put my skin there. Support libraries are supposed to make my life easier, nor fuck me about.

    2. Well, If you don’t like the tone, focus on the content.

      To be direct, implying that I’m “tearing apart” the GTK project is hyperbole; if anything, I’m hoping to make it better (though I doubt that any GTK developers will read my blog, or pay any attention to it, and it’s certainly their right to ignore me if they want to). I’m criticising the plan to break API compatibility on a regular basis, and I’m suggesting an alternative: spend some effort now on developing a long-term stable API. The indignation, in fact, comes partly from the belief that the current development plan will do the project more harm than good, and partly from concern about the flow-on effects – because, as I say, this is a foundation library we’re talking about; constantly breaking API compatibility affects other projects.

    1. Right, and as users and developers have to do that at least twice a day it is entirely unsuitable for serious work.

  3. I honestly say screw GNOME and their “brand identity”. I honestly think that’s why they keep breaking the API, because it’s meant for GNOME and it needs new “features”.

    “GTK3 isn’t a reliable API. Maybe it should be called libgnome instead.”

    “I genuinely get the feeling that GTK 3.4 is developed for Gnome 3.4, that it doesn’t really matter if it breaks things and that we’re not supposed to use it outside of Gnome.”


  4. Hi, I’d like to draw your attention to some further information about the GTK release schedule discussion, other than just the one blog post about it that you quote. It will hopefully alleviate some of your concerns. We are actually trying to address people’s concerns about the 3.18-to-3.20 breakage, not make them worse.

    – and some FAQs https://wiki.gnome.org/Projects/GTK%2B/Lifecycle/FAQ

    I definitely understand how the initial blog post made it sound. However, if you read the proposal and come discuss it at the mailing list thread https://mail.gnome.org/archives/gtk-devel-list/2016-June/thread.html#00029, you’ll have a chance to influence the decision with your opinion, much more effectively than with a hot take on your blog. I look forward to seeing you there!

    1. Thanks for your comment. I’ve now sent en email to the gtk-devel list, as you suggested. It’s currently pending moderator approval. I wouldn’t in all honesty normally expect that an outsider stepping in with what is essentially a criticism of a group decision would go down extremely well, which is large part of the reason I didn’t bother initially, but maybe that’ll not be a problem in this case. We’ll see, I guess.

    2. Ok, it’s been several days and it seems my email hasn’t been approved. I’ll duplicate here for the record:


      I’m not a regular poster to this list and am not subscribed. I’m posting here now at the suggestion of Philip Chimento, who read a blog post I wrote recently [1] about the GTK+ lifecycle plans that have recently been discussed/announced. This blog post of mine received, for whatever reason, a lot more attention on anything than my blog normally does (just shy of 20,000 views in a day, when my average views per day of the whole blog is around 10) – which puts me in a position such that I feel I should address the concerns in a more head-on (and level-headed) fashion. I’ll freely admit the tone in the blog posting is that of an indignant rant, though I hope no-one is personally offended by it since that certainly wasn’t my intention.

      I’m concerned at the outset that this will be dismissed by people who are, after all, much more involved in GTK+ than I’ve ever been, so I ask that you read this with an open mind and with the understanding that I’m writing this out of genuine concern that’s developed from the experience of dealing with API breakages in various packages over a long time-frame.

      My concern is that the new plan lays a foundation for breaking the API on a more regular basis, and formalises the notion that this is ok if it allows innovation in the design and development of GTK. While I am all for innovation, I am disturbed that casual API breakage in its name is being legitimised, especially because I do not believe that is necessary. From the proposal [2], statements such as:

      > If new features conflict with existing ones, API might be removed, rather than being deprecated.
      … are of particular concern. Further, I don’t believe that this is consistent with the very next point in the same document:

      > there will be an effort to limit breakage as much as possible.
      “As much as possible” is a very strong statement. If this was really the aim, there would be hardly any legitimate reason to break API at all, making the whole plan unnecessary. I’d suggest that the latter point is somewhat disingenuous (even if unintentionally so); putting it bluntly, only one of those two points can be correct, and from the tone of the document, I’m assuming that it’s the first one and the second is a case of overly-strong wording. But on the other hand, removing API instead of deprecating is most definitely a breaking change that can often be *easily* avoided, let alone “as much as possible”.

      Please do not assume I am basing my entire concern on that point alone. The general feeling from the proposal and the GTK development blog post that I initially responded to is that API stability is very much being pushed to a low priority compared to the need to push development ahead. I understand that API breakage does sometimes need to occur, but I think this attitude is not a good one. In particular I think the implied notion that API stability and good development progress cannot coexist is incorrect.
      I would be reassured perhaps if the plan discussed what software development methods were in place that would help to reduce the amount of API breakage needed. What is the procedure for API design in GTK – if there indeed is one? Are APIs reviewed before they are implemented? Are they made available for public review? Are there any guidelines? The Best Practices document [3] doesn’t even mention API design; why not? Are the APIs even designed at all? I am left with the worrying impression (hopefully not correct) that the API simply emerges from the code. This would certainly explain why API breakage might be needed on a regular basis, but that’s a symptom of an imperfect design practice; if it is being found that the resulting APIs are not extensible or flexible enough for ongoing development and future requirements, then I strongly suggest that the better solution is to improve the API design methodology, rather than simply to legitimise casual API breakage.

      I understand how open-source development often works in ways that don’t suit more traditional rigourous approaches, but I would recommend establishing a set of guidelines for API development, and at a minimum have each commit/check-in that introduces new API be reviewed against those guidelines by a developer not otherwise involved. Since I can’t claim to know how GTK development proceeds there’s the possibility that something like this (or better) already happens – in which case, perhaps my concerns can be dismissed; if so, please consider publishing details and updating the Best Practices document.

      Thanks for reading,


      [1] https://davmac.wordpress.com/2016/07/05/why-do-we-keep-building-rotten-foundations
      [2] https://wiki.gnome.org/Projects/GTK%2B/Lifecycle

  5. Hi, thanks for following up with the list. I’m sorry the moderation is taking so long; I’ve posted on the list to bring it to a moderator’s attention.

    > I wouldn’t in all honesty normally expect that an outsider stepping in with what is essentially a criticism of a group decision would go down extremely well, which is large part of the reason I didn’t bother initially, but maybe that’ll not be a problem in this case.

    I hope our community is better than your expectations. In any case, it’s a group proposal, not a group decision, yet, and I believe people will consider input, whether from outsiders or no, to be helpful in the process of turning the proposal into a decision.

      1. FWIW I think the other emails you refer to are from list subscribers. My email requires approval from a moderator because I’m not subscribed to the list. I suspect that in this case either the moderator has gone AWOL (or is not reading their email, or the notification went into their spam folder, etc), rather than there being any deliberate lack of approval. It’s not great but it’s very different from there being a deliberate refusal to accept the message.

    1. .NET has pretty good API (and ABI) stability, actually. A lot of stuff from .NET 1.0 days compiles just fine on 4.6, for example. At the very least, within a single major version, things don’t “just break” (and if they do, it’s treated as a bug).

      .NET Core was a major break, but it also follows semver, in that you can stick to a given major version, and expect API stability.

  6. @Vanguard: .NET framework breaks ABI every 6 months? Since when? At any rate, I only seem to have two full frameworks installed per bitness here on Windows 7 (one stuck at ~2.0, and another at ~4.6); this seems roughly analogous to GTK now. (Granted, there are some partial ones too, but that all seems to be either build tools or legacy libraries.)

  7. Looks like there’s some additional information at https://blog.gtk.org/2016/09/01/versioning-and-long-term-stability-promise-in-gtk/

    It’s rather wordy, but the meat is contained here: “Updates within long-term stable series will be ABI stable. Alongside these stable series, GTK+ development will continue in semi-stable development series. These development releases will contain some API changes between minor versions, although changes will be limited wherever possible.”

    So no semantic versioning, which is rather sad.

    1. Yes; they greatly improved the versioning scheme after a strong community response. I think this is really good and shows the developers are listening, even if they didn’t adopt “full” semantic versioning. (Note that if you pretend the x.y.9z releases don’t exist, then you do in fact have semver*).

      I wouldn’t say it fully addresses all my concerns (they are still, in my eyes, needlessly removing “stable” API between stable cycles), but it’s an improvement.

      (*: well, assuming that they don’t do anything silly with odd-numbered minor releases. The release plan makes no mention of them, so I’m assuming they won’t do them; I think it’s technically still semver if you skip numbers).

  8. I’m a bit confused about the debate, trying to determine which side is “right”.

    it seems that the GTk developers are stating that GTk3 should be considered unstable. That being the case, application developers should target GTk2.

    There’s an inherent conflict between “new and shiny” and “old and stable”. It’s up to the app devs to decide which way they want to go. It seems that the app devs wanted “stable”, but opted for “shiny”. They made the wrong choice.

    Personally, if I were building an app, I’d want to go with a known quantity that would be compilable a decade into the future and that I didn’t have to keep tweaking every 6 months. That means GTk2. It’s true that I don’t get all the latest and greatest, but then that’s the decision you have to make.

    So really, I’m not so sure it’s the GNOME teams “fault”, as app devs making the wrong decision.

    1. @mcturra2000: You are indeed confused. GTK+ 3 is stable. No new application should be developped in GTK+ 2.

      The developers that want the “new and shiny” will focus on GTK+ 3.9x (this is what will become GTK+ 4).
      The developers that want the “old and stable” will focus on GTK+ 3.22 (last version of GTK+ 3).

      Once GTK+ 3.9x will be stable enough, the next version will become GTK+ 4.0, the API will be frozen, and it will be the new stable.
      Then GTK+ 3 will then be able to retire (at minimum) 3 years after its last stable version.

      Then after a few releases of GTK+ 4.x, a new API-breaking branch will be open, GTK+ 4.9x which will in the long run give birth to GTK+ 5.

      So you will always be able to chose the version that best serves your goals (stable or bleeding edge). This will avoid the problems seen in the CSS3 theming engine in GTK+ 3 which was under heavy development and couldn’t provide compatibility through versions. This kind of “step-by-step” development can’t be avoided, and deserves to a “bleeding edge” branch, so it doesn’t cause headaches to people that prefer stability. This harsh experience of GTK+ theme writers is what lead to the new release scheme.

      BTW, thinking you can build an app once for all for the next 10 years is completely unrealistic. This is called an unmaintained project. GTK+ 2 won’t be maintained forever, so shortly after GTK+ 4 will be released, GTK+ 2 will probably phase out. Maintainers can’t clone themselves, and each branch adds overhead, so they most probably will maintain 2 stables (GTK+ 3 and GTK+ 4), not 3. As a result, distributions will slowly get rid of it as they did with GTK+ 1.2, and get rid of software built with it if it’s not updated. They don’t like stuff that doesn’t get security fixes.

      Sure you’ll still be able to build from source, but the same is true for GTK+ 3. So there is no good reason (except some specific multiplatform issues with Windows support) to base a new application on GTK+ 2. And the same is true for all software I think. It’s like telling people in 2018 to target Windows XP as an OS as it’s more “stable”.

  9. With some (not so recent) choices in FOSS, you have to wonder if part of the plan is job security and company entrenching. So much rewriting, going for huge and overcomplicated code bases, with big webs of dependencies… perfect to stay busy and disallow competition (are small teams going to fork Gtk 3.18, eg, and mantain it for their own small project? no, of course not).

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.