Rails devs: “we don’t know how to fix it, therefore it’s not a bug.”

As per title:


I really don’t know whether to laugh or cry. It would certainly be funny if

  1. I didn’t actually have to use Rails
  2. I hadn’t spent several hours debugging the issue –
  3. – and producing an executable test case, like they asked me to
  4. – which required setting up a whole new Ruby environment, because the test case template didn’t want to work with JRuby which is what we normally use.

This response is just a joke:

empty? removed the select values because it needs to call count(:all) or it will fail in some conditions.

I’d love some more specifics on why it needs to call count(:all) and what these alleged fail conditions are. In any case, why does adding ‘count(all)’ require throwing the other select values away? Just use the original query as a subquery and count the result rows. Or, don’t perform a count at all – just test whether the query returns any results. I mean this is not a difficult problem. Are the Rails developers really this incompetent?
Also, does the following code (from activerecord) really make sense to anyone?

  select_values = [
  select_values += select_values unless having_values.empty?

  select_values.concat group_fields.zip(group_aliases).map { |field,aliaz|
    if field.respond_to?(:as)
      "#{field} AS #{aliaz}"

  relation = except(:group)
  relation.group_values  = group
  relation.select_values = select_values

I mean, keeping in mind that I know the purpose is to build up an SQL query, most of it makes a hazy sort of sense. But what is this line:

  select_values += select_values unless having_values.empty?

I just can’t figure out for the life of me why you’d ever need to list every column in a ‘select’ statement twice, and why there being any ‘having’ clause would remove this need. And the real clincher is, there’s not a single comment in the code to explain why this would be necessary.

I’m starting to feel like using the Rails framework was a mistake.

Update 3/1/2015: Blergh, bug closed due to inactivity (they couldn’t be arsed fixing it – which could have been done with a simple documentation update – and so now the bug disappears). I’m done with reporting bugs to the Rails devs. If I encounter a serious bug in the future, I’ll fork and fix it myself. Code quality and general process in Rails is just awful.


Java’s Nimbus look-and-feel and custom keymaps

I recently discovered that the Nimbus look-and-feel in Java ignores background colour that’s been set using setBackground() on a JEditorPane. Apparently this is not a bug; look-and-feels are allowed to ignore the colour that you specify with setBackground() (though this raises the question of why the method even exists). With Nimbus, it turns out, you can control background painting by setting a “client property” on the editor pane (using putClientProperty()) – specifically, you set the “Nimbus.Overrides” property to a UIDefaults object, which is essentially a property-value map. The “EditorPane[Enabled].backgroundPainter” property is supposed to be set to a BackgroundPainter which Nimbus will then use, though if you set it some other object type (such as a String or even a Color) it will ignore the property and instead paint the background using the colour that was set with setBackground() – which of course raises the question of why it couldn’t do this in the first place, but, meh, whatever; at least we have a solution. (Any properties which Nimbus needs that aren’t in your provided UIDefaults will be inherited from Nimbus’ default property set).

There is (of course) Only One Problem.

Setting the “Nimbus.Overrides” client property apparently prevents any custom keymaps that you might have assigned to the JEditorPane from working. This is true even when the UIDefaults table you provide is empty, implying that the mere act of assigning something to the Nimbus.Overrides property is changing behaviour of the look-and-feel, which is surely broken.

I’ve submitted a bug to Oracle (via bugs.java.com) though the hit rate on getting a response for them hasn’t been 100%, so we’ll see what happens. Once you submit you get the following text:

What’s Next?

We will review your report. Depending upon the completeness of the report and our ability to reproduce the problem, either a new bug will be posted, or we will contact you for further information. You will be notified by email in either case.

… (emphasis mine) – however I’ve submitted reports in the past which got no response. I will update this blog entry if/when I receive an acknowledgement. (A few minutes later – first update! I’ve received an automated email stating that “We are evaluating this report and have stored this with an Review ID: JI-9012303”).

Update 24/05/2104: I still haven’t heard anything, but today I discovered this. So, it looks like this was recognised as a bug, but I, the initial reporter, was never notified at all – this seems like poor form. At least my report wasn’t wasted.

Test case below.

Continue reading

Custom memory allocation is not possible in standard C

I’ve recently been perusing the C’11 standard final draft, mostly hoping to find some resolutions to the various inconsistencies and problems I’ve noted previously with the C99 standard (with no real success). In particular I read section 7.22.3 (C11; 7.20.3 in C99), which discusses the malloc family of functions:

The pointer returned if the allocation succeeds is suitably aligned so that it may be assigned to a pointer to any type of object and then used to access such an object or an array of such objects in the space allocated …
This little snippet is interesting because it hints at the intended usage of the methods. We call malloc (for instance), which returns a “void *”, and then we assign it to some other type of pointer and use that pointer to access the object that we allocated. What I begin to wonder about is how well this conversion of pointers is defined.
malloc returns a “void *”. When we assign a “void *” value to a pointer of another type, C99’s says the value of the “void *” is “… converted to the type of the assignment expression and replaces the value stored in the object designated by …” the assignee. Fine; so what happens during the conversion? First,
A pointer to void may be converted to or from a pointer to any incomplete or object type. A pointer to any incomplete or object type may be converted to a pointer to void and back again; the result shall compare equal to the original pointer.
Note with care “the result shall compare equal to the original pointer”; 6.5.9p6 tells us what this means:
Two pointers compare equal if and only if both are null pointers, both are pointers to the same object (including a pointer to an object and a subobject at its beginning) or function …
(Plus a couple of other cases, to do with arrays). What’s interesting here is the stipulation that the pointers must compare equal, not that they are the same in other regards – a bit more on this later. What’s also particularly interesting is the notion that the situations for which pointers, after conversion, must compare equal and therefore point at the same object are quite limited. Other than (as quoted earlier) we have only
A pointer to an object or incomplete type may be converted to a pointer to a different object or incomplete type. If the resulting pointer is not correctly aligned for the pointed-to type, the behavior is undefined. Otherwise, when converted back again, the result shall compare equal to the original pointer. When a pointer to an object is converted to a pointer to a character type, the result points to the lowest addressed byte of the object. Successive increments of the result, up to the size of the object, yield pointers to the remaining bytes of the object.
It’s not even made explicit that conversion from a “T *” (for some type T) to “char *” results in a pointer which must compare equal to the original, i.e. it’s not perfectly clear that the “lowest addressed byte” of is the same thing as a “subobject at its beginning” as per 6.5.9p6 (though this case it seems a reasonable assumption). What’s particularly interesting is that conversion from a “void *” to some “T *” is not particularly well-defined. All that we have is, which says that the conversion is permissible, and that if inverted the result will compare equal with the original pointer (No no no! It’s the other way around – converting from T* to void* and back will compare equal; from void* to T* and back need not). Take the following code for example:
int a = 5;      // 1
void *va = &a;  // 2
float *fa = va; // 3
void *vb = fa;  // 4
float *fb = vb; // 5
int *pa = vb;   // 6

Now answer the following questions, assuming that the alignment requirements for all involved types are met (except when answering the first question). Write down your answers:

1. Is it possible for line (3) to produce undefined behavior, if the alignment requirements for ‘float’ are more stringent than that for ‘int’? (assume that line 3 does not produce undefined behavior for the following questions).

2. Would (fa == va) necessarily be true immediately after (3)?

3. Would (vb == va) necessarily be true immediately after (4)?

4. Would (fb == fa) necessarily be true immediately after (5)?

5. Would (pa == &a) necessarily be true immediately after (6)?

Here are my own answers, with discussion:

1. No, because allows conversion from a void pointer to any type of pointer. The conversion is not necessarily allowed in both directions, however. does not come into play because a pointer to void is not a pointer to an object type.

Yes, I think, because comes into play. This does open the question of what the purpose of is, however, since it appears to be completely encompassed by (One could perhaps be forgiven for thinking that is trying to remove the alignment requirements in the case of conversion from the “void *” type).

2. No. Nowhere does it state that these pointers must compare equal.

3. No.  As per costeau’s comment below, the transition from void-pointer-to-object-pointer-to-void-pointer is not guaranteed to yield a pointer equal to the original.

Yes. We converted a “void *” (va) to a “float *” and back again, so by (and p7)  the result must compare equal with the original. This shouldn’t be surprising.

4. Yes, by p1. Again, this shouldn’t be surprising.

5. No.This is the really disturbing result.Although va and vb must compare equal (and therefore point at the same object), no further requirements are placed on the similarity of their behavior. (actually there is no requirement that va and vb compare equal! I have really goofed this up). Thus, converting them both to “int *” might yield different pointer values which do not compare equal (and which do not point at the same object!). Putting it another way, although va and vb must compare equal (wrong…), they need not be the same value!

Edit: A much better example, because I got that so wrong before:

int a = 5;
void *ap = &a;
void *bp = &a;

Note that it is not necessarily true that ap == bp, but it is true that (int *)ap == (int *)bp.

More generally, take this example:

char *a = malloc(sizeof(int));
void *r = a;
float *f = r;

Note that there is no guarantee now that f addresses the same object as does a (and indeed, there’s no guarantee that r holds a pointer equal to the one that was returned by the call to malloc!). So, if we were able to obtain a pointer to a chunk of memory (i.e. an object) there’s actually no way we could use it, unless it happened to be the right pointer type to begin with.

I really, really, do not like the answer for 5, but I can’t find a way to come to the opposite conclusion based on the standard as worded. Note that in general, conversion from “void *” to another type of pointer – in fact, conversion of any “T *” to another pointer type other than “char *” – is not very well defined. Interestingly, the only thing in the standard that lets us use the result of the malloc() function is the documentation for malloc() itself (as quoted earlier). But creating a custom, general-purpose memory allocation (malloc-like facility) is just not possible. You simply can’t return a usable “void *” value unless you derived it from a pointer to the desired type in the first place. Am I missing something?

What “restrict” really means

I recently stumbled across a web page claiming to be “Demystifying The Restrict Keyword“, a claim which I consider dubious at best. Of the semantics of C99’s restrict, the page (written by a Mike Acton) says:

[when declaring a restrict-qualified pointer] I promise that the pointer declared along with the restrict qualifier is not aliased. I certify that writes through this pointer will not effect the values read through any other pointer available in the same context which is also declared as restricted.

Err, wrong. Firstly, a pointer declared with restrict is allowed to have aliases, it’s just that restrictions are placed on how those aliases are used or created. The C99 standard as always makes use of completely inaccessible language to describe the semantics of “restrict” – see, “Formal definition of restrict” – but in this case we can wade through the cruft and extract some actual meaning. The key point is that if a value pointed at by a restrict-qualified pointer is accessed through that pointer in any way, and is also modified through some pointer expression, then the latter pointer expression must be “based on” the restrict-qualified pointer. “Based on” means what it sounds like it means – essentially, that the value of the expression depends on the value of the pointer on which it is based. Here’s a counter-example to the restriction on aliasing expressed by Mike Acton above:

int * restrict p = ...;
int a = *p;   // here we have used 'p' for access
int * q = p;  // q aliases p
*q = 4;       // q is "based on" p, so this is allowed

The second part – “I certify that writes through this pointer will not effect the values read through any other pointer available in the same context which is also declared as restricted” – is also wrong (and not just grammatically). What restrict means is (and I’m simplifying just a little) that – if the restrict-qualified pointer is used to access an object, and the object is modified through any pointer, then the latter pointer and all other pointers (regardless of whether they are qualified with restrict or not) which are used to access the object will be “based on” the restrict-qualified pointer. Because we are allowed (with some restrictions) to assign the value of one restrict-qualified pointer to another, we can easily devise another counter-example to Mike Acton’s contract:

int * restrict p = ...;
    int * restrict q = p;  // q aliases p, both are restricted
    // Note that both q and p are "available in [this] context"
    *q = 4;  // this writes through q and affects the value read through p,
             // and yet is perfectly legal! (because q is based on p).
(*p);  // note the read through p must occur outside the block above, because
       // p is not based on q.

“restrict” does have some tricky semantics and it’s a shame that someone would “demystify” them so misleadingly. It could be a grave mistake to think that a non-restrict-qualified pointer can always safely alias a restrict-qualified pointer for instance.

It doesn’t quite stop there; the page goes on with a length example of using restrict to improve optimisation of a small function. The solution presented goes too far, assigning the address of each member of three different objects of the same structure type to different restrict-qualified pointers before performing calculations on them. This shouldn’t really be necessary; qualifying the structure pointers themselves with restrict should suffice, since the compiler should already know that different members of the same struct cannot legally alias (however, I’ve done some testing with gcc 4.6.3 and I’m not sure that this is the case with this compiler). Also, I’m fairly certain the solution invokes undefined behaviour via the odd and unnecessary pointer arithmetic involving a ‘stride’; A better version follows:

move( vector3* restrict velocity, 
      vector3* restrict position, 
      vector3* restrict acceleration, 
      float             time_step,  
      size_t            count)

  for (size_t i=0;i<count;i++)
    float* restrict acceleration_x = &acceleration[i].x;
    float* restrict velocity_x     = &velocity[i].x;
    float* restrict position_x     = &position[i].x;
    float* restrict acceleration_y = &acceleration[i].y;
    float* restrict velocity_y     = &velocity[i].y;
    float* restrict position_y     = &position[i].y;
    float* restrict acceleration_z = &acceleration[i].z;
    float* restrict velocity_z     = &velocity[i].z;
    float* restrict position_z     = &position[i].z;
    const float ax  = *acceleration_x;
    const float ay  = *acceleration_y;
    const float az  = *acceleration_z;
    const float vx  = *velocity_x;
    const float vy  = *velocity_y;
    const float vz  = *velocity_z;
    const float px  = *position_x;
    const float py  = *position_y;
    const float pz  = *position_z;

    const float nvx = vx + ( ax * time_step );
    const float nvy = vy + ( ay * time_step );
    const float nvz = vz + ( az * time_step );
    const float npx = px + ( vx * time_step );
    const float npy = py + ( vy * time_step );
    const float npz = pz + ( vz * time_step );

    *velocity_x   = nvx;
    *velocity_y   = nvy;
    *velocity_z   = nvz;
    *position_x   = npx;
    *position_y   = npy;
    *position_z   = npz;

From my tests this was by far the most performant (on a core-i7 based machine) with gcc (compilation options: -std=c99 -O3 -march=core2). When using the clang compiler, there wasn’t much variation in results between any version, including the naive version which didn’t use restrict at all (a slight performance benefit can be noticed when simply introducing restrict qualification into the parameter declarations, but no further benefit comes from using the code above, and only a negligible benefit comes from Mike Acton’s code which I believe invokes undefined behaviour anyway).


Cinnamon (again) and hacky non-bugfixes

After getting frustrated with Ubuntu’s Unity interface to the point of wanting to bludgeon myself to death with a bag of squirrels, I tried out Cinnamon by manually installing the latest version. It’s actually pretty nice, except for one major annoyance: every time I get a skype message, it un-minimises the relevant chat window and brings it over the top of my current foreground window/process. This is really annoying. There is a bug entry in Cinnamon’s bug database, but it closes with this frankly disturbing comment by someone called Clement Lefebvre (clefebvre):

I don’t like the idea of this being configurable.. mainly because this isn’t a question of preference but a question of fixing a bug. The focus should always be taken when the user launches a new app and never be taken on his behalf when he doesn’t trigger the creation of new content.

For Skype, I fixed the bug with this https://github.com/linuxmint/Cinnamon/commit/c5bbcad1cc4cc8183f2556deef867d0fae5f0109

Well, right, it’s a bug and should be fixed properly, but, check out the fix:

if (!window || window.has_focus() || window.is_skip_taskbar() || window.get_wm_class() == “Skype”)

What the … it actually checks for Skype and handles it specially. That’s got to be wrong. Clement then goes on to say:

For other apps which face a similar issue, I’d like people to insert the following code at line 27 in /usr/share/cinnamon/js/ui/windowAttentionHandler.js:

global.log_error(“Focus stolen by : ” + window.get_wm_class());

Then restart cinnamon, and when the focus is stolen from you, click on the ^ applet, troubleshoot, looking glass, click on the error tab, and you should see the wm class name of the app which stole your focus. Give me that wm class name, and we’ll add it to Cinnamon.

Oh for fuck’s sake… this is not how you do these things. Fix the bug in Cinnamon, make it respond to notifications properly for all apps; don’t just implement a workaround on an app-by-app basis. Am I missing something here? Surely other window managers aren’t implementing this workaround (because they don’t need to, because they handle the notification correctly in the first place)? This doesn’t exactly inspire confidence in the quality of Cinnamon.

PHP, thou art a wart on the face of the web

I’ve long despised most dynamically typed languages, for reasons which I’ve yet to properly enunciate, but I’ve begun to loathe one above all the others: yes, I’m referring to the abomination that is PHP. I’ve never got around to properly writing about why I abhor it so much but I was recently given a link to an article on another blog, which contains (amongst a much larger and very well argued discussion) this little gem (reproduced with permission):

I can’t even say what’s wrong with PHP, because— okay. Imagine you have uh, a toolbox. A set of tools. Looks okay, standard stuff in there.

You pull out a screwdriver, and you see it’s one of those weird tri-headed things. Okay, well, that’s not very useful to you, but you guess it comes in handy sometimes.

You pull out the hammer, but to your dismay, it has the claw part on both sides. Still serviceable though, I mean, you can hit nails with the middle of the head holding it sideways.

You pull out the pliers, but they don’t have those serrated surfaces; it’s flat and smooth. That’s less useful, but it still turns bolts well enough, so whatever.

And on you go. Everything in the box is kind of weird and quirky, but maybe not enough to make it completely worthless. And there’s no clear problem with the set as a whole; it still has all the tools.

Now imagine you meet millions of carpenters using this toolbox who tell you “well hey what’s the problem with these tools? They’re all I’ve ever used and they work fine!” And the carpenters show you the houses they’ve built, where every room is a pentagon and the roof is upside-down. And you knock on the front door and it just collapses inwards and they all yell at you for breaking their door.

That’s what’s wrong with PHP.

I’ve not really ever had to write much PHP but I’ve sat looking over the shoulder of someone who was in the process of coding some small PHP scripts, and I was horrified. The problems with the ‘==’ operator, and the overloading of the function return types, were particularly unsettling. I’ve written previously about my difficulty in just compiling PHP so that it worked as I wanted – which just adds to the feeling that PHP was simply slapped together from a load of unrelated pieces, and it was only by chance that a turing-complete language emerged. One of these days I’ll get around to explaining just why dynamic languages are bad, but in the meantime get some entertainment (and perhaps education) by reading Eevee’s post.

2012-05-05 addendum: Add “insecure” to my list of gripes.