initializer_list, lambdas, operators, templates, parameter packs, variadic functions…

Where to start with this one. I opened a can of worms trying to write a lambda expression. Or more specifically, a function that takes a lambda expression as an argument. This seems easy enough, right? Lots of examples everywhere. But what would we learn by taking the easy way out?

My target for this lambda-accepting function is my crude collision detector, which is going to allow every shape in my OpenGL world to call “Collide” on every other shape, exactly once, with the obvious exception that a shape cannot collide with itself. Without lambdas this looks something like this:

std::vector<std::unique_ptr<Shape>> shapes;

// ... initialize shapes, more on this later ...

for (int i = 0; i < shapes.size() - 1; i++)
{
	for (int j = i + 1; j < shapes.size(); j++)
	{
		shapes[i]->Collide(shapes[j].get());
		shapes[j]->Collide(shapes[i].get());
	}
}

The way I have written collision detection, it has to be done both ways – i.e. for two Shapes to collide, we must call a->Collide(b) and b->Collide(a). Each call represents the effect one Shape has on the other.

What I’d really like to do is have a spiffy pairwise method, one that we could invoke with a lambda like this:

shapes.pairwise([&](Shape* a, Shape* b) { a->Collide(b); });

This is much nicer than my nested for loops. It’s much clearer, too, I think; for each possible pair, collide one with the other. What’s not to like.

Well here is where the fun starts. First of all, where do we put this pairwise method? My first thought is to subclass std::vector, but OH NO YOU DIDN’T according to the internet… we are not supposed to do that. As best I can figure it’s because std::vector has a private destructor which won’t get called when the subclass gets destroyed. Ok, that sounds reasonable enough. I’m not using that many methods on the shape vector anyway, so why not just make a new class with a vector as a member and re-implement the methods I need? This is known as “composition over inheritance”. Yay, it has a name.

So we have our new class, which I have called ShapeVector. How do we initialize this thing? Just like we’d initialize a vector, right? Something like this:

ShapeVector shapes = {
	std::make_unique<BoundingBox>(glm::vec3(-3, -3, -3), glm::vec3(3, 3, 3)),
	std::make_unique<Cube>(glm::vec3(1.5, 0, 0)),
	std::make_unique<Cube>(glm::vec3(-1.5, 0, 0))
};

Nice and tidy. Remember we are using unique_ptrs to preserve polymorphism and be all modern like. What does this look like in our ShapeVector constructor? Maybe one of those good old ellipsis things like in C? No, there is a new way! It’s called an initializer_list, and it gets magically created when a construct like the above is used. Here’s how to copy stuff out of the initializer_list into our internal array:

ShapeVector(std::initializer_list<std::unique_ptr<Shape>> shapes)
{
	for (const std::unique_ptr<Shape>& shape : shapes)
	{
		this->shapes.push_back(std::move(shape));
	}
}

Real simple right? We’re being all careful about const and & and move. There’s only one problem, though… it doesn’t compile. You’ll get an ‘attempting to reference a deleted function’ error.

Here’s where you can go down a rabbit hole finding all kinds of hacks on the internet that don’t work. My advice – forget about this approach. Whoever created this initializer_list thing clearly did not want it to be used this way, so let’s just let them have that.

It turns out there’s this new ‘variadic function template’ thing you can do that is like the new version of the C ellipsis, here’s what that looks like:

template<typename... Args>
ShapeVector(Args... args) {
	addshapes(args...);
};

template<typename T>
void addshapes(T& shape) {
	shapes.push_back(std::move(shape));
}

template<typename T, typename... Args>
void addshapes(T& shape, Args&... args) {
	shapes.push_back(std::move(shape));
	addshapes(args...);
}

Now I’m not going to lie to you here, I don’t totally understand all this template/typename stuff. I copied it off the internet. What I think this code is saying is that you can pass in a bunch of anything you want, and I’m going to try to stuff it into the shapes vector whether it’s the right type or not. But since I’m me, and I know what I’m going to pass in, we’re going to sidestep that issue for the moment.

One notable thing about this new construction is the recursive nature of addshapes. Apparently there is no way to just iterate through this args thing! They forgot to make that part. I guess I am glad they didn’t put it in an initializer_list.

Also of note is that we had to make the addshapes method because we can’t do this recursive business with constructors. And, all this template junk has to be in the header file, because reasons. I am beginning to think this is a language for people who like a lot of intricate and mysterious rules. Because who doesn’t, really?

Anyways, our initialization now looks like this:

// create vector of shapes
ShapeVector shapes(
	std::make_unique<BoundingBox>(glm::vec3(-3, -3, -3), glm::vec3(3, 3, 3)),
	std::make_unique<Cube>(glm::vec3(1.5, 0, 0)),
	std::make_unique<Cube>(glm::vec3(-1.5, 0, 0))
);

And that isn’t so bad, really. Parentheses instead of curly braces, and no equals sign. Heck I think it looks better, if you want my honest opinion.

Weren’t we doing some kind of lambda thing? Ah yes, here is what that looks like – also a template, in the header file.

template<typename Func>
void pairwise(Func pairfunc)
{
	for (int i = 0; i < shapes.size() - 1; i++) {
		for (int j = i + 1; j < shapes.size(); j++) {
			pairfunc(shapes[i].get(), shapes[j].get());
			pairfunc(shapes[j].get(), shapes[i].get());
		}
	}
}

Again, I think this template stuff is saying that I can pass in any old function I want, and we’re just going to go tits up if it’s not the right signature. That seems unfortunate. Perhaps we will find a way to fix that later. But for now, this allows me to do the pairwise lambda call I described above, so I’m happy.

I also made a foreach call that looks like this:

template<typename Func>
void foreach(Func eachfunc)
{
	for (auto& shape : shapes) eachfunc(shape.get());
}

That one wasn’t as satisfying, because I’m basically turning a one-liner into another one-liner. Let’s do something interesting – let’s define an operator! How often do you get to do that in C++. Let’s define the [] operator, since that makes sense for a vector-esque class.

Shape* operator[](int i)
{
	return shapes[i].get();
}

This is really basic stuff, but it’s exciting to me, okay? If that is not interesting enough for you, did you know that if you have an array of ints you can reverse the operands for [] and it does the same thing? Like, if your array is called ints, you can reference ints[5] or 5[ints] and you’ll get the same result. There, now you know.

Initializers – whoops!

I mentioned initializers yesterday, here’s what I’m talking about:

std::vector<Shape> shapes = {
	Cube(glm::vec3(1.5, 0, 0)),
	Cube(glm::vec3(-1.5, 0, 0))
};

This stores objects instead of references, and then the for loop looks like this:

for (auto &shape : shapes)
{
	// do stuff with shape
}

You can take out the &, too, and I’m going to be brutally honest with y’all I don’t know what the difference is.

Don’t matter though, because guess what – this doesn’t work! All of the Cubes act like Shapes, even though I’ve dutifully marked the functions as virtual. Why is this? Turns out there is something called “Object slicing”. When I assign a Cube to a Shape, The piece of Cube that makes it different from Shape gets sawed off, and there’s just a Shape left that doesn’t know it was ever a Cube. Why would anyone ever want to do this? You’d think at least there’d be some kind of ‘slice’ keyword so that you have to tell the compiler, yes, I really want to do this nonsensical thing. It would keep a lot of people out of trouble, not least of all myself.

This begs for an experiment, though, that may unravel some of the mysteries of &! Here’s my theory. Since object slicing is a thing, then if I have an array of Cubes and I do this:

for (Shape shape : cubes)
{
	// do stuff with shape
}

The cubes will get sliced, and act like Shapes. But if I do this:

for (Shape &shape : cubes)
{
	// do stuff with shape
}

The cubes should not get sliced, and act like Cubes! Because it’s a reference, I’m not actually copying the object! Right?? Do I know my &??

It turns out I do, but there’s an even more bizarre side effect that is, in retrospect, obvious… if you do anything in this for loop (the first one, without the &) that affects the state of the object, it only affects the copy, not the original one in the array! So in that situation the & really is your friend, in many ways.

This is all fascinating, but I’m going back to pointers so that I can have my polymorphism.

Vectors and stuff

I have ventured into class hierarchies in my OpenGL adventure. At the top is the “Shape” class, which knows how to supply a list of vertices and colors for OpenGL to consume. Then there is the “Cube” subclass, which, as the name implies, is a cube.

Using my old C/C++ instincts intertwined with newfangled C++, I declared my vector of objects like this:

std::vector<std::unique_ptr<Shape>> shapes(2);  // only 2 shapes so far

And then I filled in the array like this (my cube constructor takes a vector as initial position):

shapes[0] = std::make_unique<Cube>(glm::vec3(1.5, 0, 0));
shapes[1] = std::make_unique<Cube>(glm::vec3(-1.5, 0, 0));

I wanted to be super modern, so I tried iterating over it like this:

for (auto const shape : shapes)
{
    // do stuff with the shape
}

But I got another one of those gibberish warnings about a deleted function. It turns out, unsurprisingly enough, that every time an element of shapes is assigned to shape, it thinks someone is trying to steal a unique_ptr. However, there’s a handy little solution:

for (auto const& shape : shapes)
{
    // do stuff with the shape
}

The & operator was always kind of mysterious to me, like it’s kind of faking a pointer but not really. But I am going to try not to think about it too hard, and just accept that this works. Apparently using a reference does not amount to stealing.

It turns out I could have done this whole thing differently with initializers! But that is for another day. Probably tomorrow.

unique_ptr, move, auto

I have learned my first fun little lesson with C++.

OpenGL involves buffers full of floating point numbers, or GLfloats, both for vertices and colors. These buffers have to be passed into OpenGL functions as ‘raw’ buffers, i.e. as a GLfloat* that points to a bunch of consecutive GLfloats.

So this seems like a perfect opportunity to create a dynamic array using some of the funky new C++ pointer types! This should get me in some trouble. I could probably just use a vector, but that wouldn’t be any fun. Anyway, here’s my class:

#include <glad/glad.h>
#include <memory>

class DynamicGLFloatBuffer
{
	int len;
	int capacity;
	std::unique_ptr<GLfloat[]> data;

public:

	DynamicGLFloatBuffer();
	void append(const GLfloat* newdata, int newdatalen);
	void clear();
	GLfloat* get();
	int length();
	int size();
};

And the constructor:

DynamicGLFloatBuffer::DynamicGLFloatBuffer()
{
	len = 0;
	capacity = 65536;
	data = std::make_unique<GLfloat[]>(capacity);
}

There’s the fun part, that make_unique function. That creates a unique_ptr, which ensures that the memory allocated to the array belongs to us and only us, and it will be deleted for us when we go out of scope.

Where I got into trouble was in the append function:

void DynamicGLFloatBuffer::append(const GLfloat* newdata, int newdatalen)
{
	// if we don't have enough room
	if (capacity < len + newdatalen)
	{
		// double capacity until we have enough
		while (capacity < len + newdatalen) capacity *= 2;

		// create new array
		std::unique_ptr<GLfloat[]> tempdata = std::make_unique<GLfloat[]>(capacity);

		// copy data into new array
		std::memcpy(tempdata.get(), data.get(), len);

		// adopt new array
		data = tempdata;
	}

	// copy memory into place
	std::memcpy(data.get() + len, newdata, newdatalen * sizeof(GLfloat));

	// adjust length to include newdatalen
	len += newdatalen;
}

You can’t see it here, but in my IDE there’s a red squiggly line under the equals sign in data = tempdata;, and it says:

function "std::unique_ptr<_Ty [], _Dx>::operator=(const std::unique_ptr<_Ty [], _Dx> &) [with _Ty=GLfloat, _Dx=std::default_delete<GLfloat []>]" (declared at line 2055 of "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.26.28801\include\memory") cannot be referenced -- it is a deleted function

Wow I am so glad they have made my life easier with these new C++ constructs.

Well it turns out you can’t just assign a unique_ptr; you have to “move” it, like this:

data = std::move(tempdata);

And then everybody’s happy. I’m not sure why they couldn’t have just overridden that ‘=‘ operator to do a move, since that’s what I have to do anyway… there’s nothing else to do in that situation.

We can go a step further into modern C++ and use the “auto” keyword to declare tempdata:

auto tempdata = std::make_unique<GLfloat[]>(capacity);

As far as I can tell, this “auto” keyword exists solely so that I won’t have to type std::unique_ptr<GLfloat[]> again. Which is nice, I guess. Seems like overkill, I’m not sure if it really makes code more readable or safer. But maybe there is something I don’t know about it yet.

React Router and…. OpenGL

I added React Router to the Ledger, which I think is quite nice. That is something I was going to have to learn at some point to become a respectable React programmer, and it is pretty straightforward.

Unfortunately I have to put the Ledger on the back burner for a couple of weeks, because I have a job interview that involves C++ and I need to brush up on my C++! It has been a while since I’ve used it professionally.

I figure what better way to make this fun than to do some OpenGL, which I have some familiarity with. I may or may not be able to display this on the web page at some point… I have done some reading about how to run C++ in a browser and it seems like more than I want to take on just for this interview. OpenGL is enough to take on! Really my objective is to use OpenGL as a way to discuss some esoteric C++ concepts that I should be able to answer questions about. I have a list of them from my friend Dave, who is a C++ guru. He has suggested the following basic topics:

  • Casting operators (static_cast, const_cast, etc)
  • Conventions for using auto keyword
  • Elementary operations on vector and other containers
  • Lambdas
  • Standard library pointer classes (auto_ptr, unique_ptr, etc)
  • Move semantics

So, I will discuss these topics as it relates to doing fun things in OpenGL, and see how far I get. As a starting point I am using Glitter, which got me up and running pretty quickly, and I am moving through the beginner tutorials at http://www.opengl-tutorial.org/. These tutorials are good because they don’t lay out the source code in its entirety… they give it to you little snippets at a time. That way you have to figure out where the snippets go, and it ensures you are paying attention instead of just cutting and pasting.

I am skipping all of the texturing and lighting stuff because that is not really important right now – all I am looking for is to draw some cubes, and move around and look around with the keyboard & mouse. Once I have that up and running we can do some fun stuff shooting cubes around and whatnot.

Jenkins!

I have gotten Jenkins up and running, which is kind of fun. It can run in tomcat which is convenient since I have already set that up, and I’ve already dealt with the permissions of tomcat publishing stuff to html directories. For anyone who is curious, I did that by creating a www-tomcat group and making the tomcat and www-data users members of it.

I found that the “pipeline” option in Jenkins was much easier to get going with than creating a freestyle project. The one thing I got hung up on was this error when trying to copy stuff out of the workspace:

cp: cannot create regular file '/path/to/filename': Read-only file system

That’s kind of frustrating, because I know it’s not a read-only file system! Thanks to this fellow, I discovered that tomcat is put in a ‘Sandbox’ so it can’t write anywhere except for log files and such. Good news, this is solvable! Unfortunately the proposed solution didn’t work for me; there was no /etc/systemd/system/tomcat9.service.d/ directory, and when I created one, it was ignored.

With a little detective work of my own I discovered that there is a file at /lib/systemd/system/tomcat9.service that contains a bunch of ReadWritePaths directives, so why not just add mine there? And voila, it works.

So now all I have to do is write the rest of my ledger program! But at least I will not be spending time copying files around.

Hooks

In the course of trying to put together this ledger application, I wanted to have a nice looking table with drag-and-drop functionality. I figure React should be really good for this, with some Bootstrap styling, whammo you have a nice looking table.

After looking through some reviews and such, I decided to go with react-table since it seems to be the most widely used, and supports drag and drop. The only catch is it’s written with “Hooks”, this newfangled React thing. The branding on this is huge. Everything is going Hooks. So I wire up this react-table example in my Ledger component, and guess what! Hooks don’t work with class components. Everything is going Functional. This is starting to feel a little forced! Like they’re just trying to create a secret club that everyone now has to learn the passcode for.

I have to learn the passcode, though, if I am going to have a pretty table. So I make a functional react-table component, and now there’s an infinite loop! Have you seen this marvel?

Error: Maximum update depth exceeded. This can happen when a component repeatedly calls setState inside componentWillUpdate or componentDidUpdate. React limits the number of nested updates to prevent infinite loops.

Okay, so the first thing I realize is that I’m setting state inside useEffect() like this:

useEffect(() => {
	$.ajax({
		type: 'get',
		url: props.parent.getConfig().baseURL + 'ledger/'
	}).done(function (ajaxData) {
		setTransactions(ajaxData);
	}).fail(function (jqXHR, textStatus, errorThrown) {
		props.parent.showAlert('Server Error', 'Server returned a status of ' + jqXHR.status);
	});
});

Did you see what I did wrong there? That’s right, the first little secret… the empty array as the second argument.

useEffect(() => {
	$.ajax({
		type: 'get',
		url: props.parent.getConfig().baseURL + 'ledger/'
	}).done(function (ajaxData) {
		setTransactions(ajaxData);
	}).fail(function (jqXHR, textStatus, errorThrown) {
		props.parent.showAlert('Server Error', 'Server returned a status of ' + jqXHR.status);
	});
}, []); // <-- that little sucker right there

Drove myself crazy trying to figure that one out, trying to stop the infinite loop with a flag in the state (nope) or load the data in the parent component (nope again). That infinite loop will find you.

So I’m all proud of myself for figuring this out, I fire up the app again and:

Error: Maximum update depth exceeded. This can happen when a component repeatedly calls setState inside componentWillUpdate or componentDidUpdate. React limits the number of nested updates to prevent infinite loops.

What!!!! I just solved that problem!!! Well it turns out you have to do this in useMemo too, which react-table uses:

const data = React.useMemo(() => transactions, [transactions]); // <-- that guy there
		
const columns = React.useMemo(() => [
	{ Header: 'Date', accessor: 'transdate' },
	{ Header: 'From', accessor: 'sourcename' },
	{ Header: 'To', accessor: 'destname' },
	{ Header: 'Note', accessor: 'comments' },
	{ Header: 'Amount', accessor: 'amount' },
	{ Header: 'Balance', accessor: 'balance' },
	{ Header: 'Reconciled', accessor: 'reconciled' }
], []); // <-- and this little sucker right here

Notice that I didn’t use an empty array for [transactions], since that actually changes. If I use an empty one there, the transactions never make it into the table.

I hope you’ve enjoyed this edition of what John learns in an evening of whacking his head against a brick wall. Maybe it has saved you some trouble.

CSRF Token Hell

When I first paired up Django + React, I couldn’t figure out the whole CSRF token thing, so I avoided the whole issue by just using GET requests. Well that is no way to earn a living so I decided to figure it out today. Holy hell!

Everyone tells a nice little story about how this is supposed to work. You’ll see this getCookie method that looks something like this:

getCookie(name) {
	var cookieValue = null;
	if (document.cookie && document.cookie !== '') {
		var cookies = document.cookie.split(';');
		for (var i = 0; i < cookies.length; i++) {
			var cookie = $.trim(cookies[i]);
			if (cookie.substring(0, name.length + 1) === (name + '=')) {
				cookieValue = decodeURIComponent(cookie.substring(name.length + 1));
				break;
			}
		}
	}
	return cookieValue;
}	

And then you’ll see some ajax configuration stuff like this:

$.ajaxSetup({
	beforeSend: function(xhr, settings) {
		if (settings.type=='POST') {
			xhr.setRequestHeader("X-CSRFTOKEN", self.getCookie('csrftoken'));
		}
	}
});

And then boom it works, right? No, you also have to add some stuff to your settings.py that looks like this:

CORS_ALLOW_CREDENTIALS = True
CORS_ALLOW_HEADERS = list(default_headers) + [
    'X-CSRFTOKEN',
]
CORS_ORIGIN_ALLOW_ALL = False
CORS_ORIGIN_WHITELIST = (
       'http://localhost:3000',
)
CSRF_TRUSTED_ORIGINS = ['localhost:3000']
CSRF_COOKIE_SAMESITE = None
CSRF_COOKIE_SECURE = False
CSRF_COOKIE_HTTPONLY = False

And then some middleware like this:

MIDDLEWARE = [
    'corsheaders.middleware.CorsMiddleware',
    ... bla bla bla standard middleware ...
]

But it still doesn’t work! Oh wait you forgot the @ensure_csrf_cookie decoration on your view, and did you try adding csrfmiddlewaretoken to your POST data, or maybe you need to call django.middleware.csrf.get_token(request)in your view… and if it still doesn’t work you can always forget the whole thing and use @csrf_exempt which is looking pretty tempting right now because you’ve been scouring the internet for five hours trying to figure out what this thing’s problem is.

AND THEN finally you come across a stack overflow post with a little green checkmark and it’s the one…

    xhrFields: {
       withCredentials: true
    }

Add that little sucker to your ajax request. And finally you can go to bed.

Scores!

I have come up with some official metrics on how the various moodilyzer schemes are doing, using the f1 score. This is a pretty widely used thing, and once again scikit makes this really easy, so I went with it. Here are the results:

moodLSTMELMo+LRELMo+MLP
grateful0.130.19
happy0.33
hopeful0.190.200.29
determined0.040.320.51
aware0.170.34
stable0.080.080.38
frustrated0.09
overwhelmed0.27
angry
guilty
lonely
scared
sad0.33
F1 scores

These are pretty low numbers, to be sure. We had a very small dataset. If we average these up for each classifier, we get:

LSTM: 0.07
ELMo+LR: 0.09
ELMo+MLP: 0.14

That’s kind of a nice result! Things improved as we went along and tried different approaches. If we use the best approach for each mood and average those out, it goes up even further to 0.21!

So, I think I am going to call it a day on text classification until I can find another fun dataset.