Multiple Outputs Bad

As promised I have attempted a neural network with multiple outputs. This is a misnomer on my part I think, because it is still a single output, just with multiple possible values. Instead of a 0 or 1, the network outputs a number from 0..12, one value for each of the possible “moods” of the input sentence.

I didn’t have to change much to get this to work. I changed the output size from 1 to 13, and then I had to make one little tweak to get it to match the example I was copying. Let’s see if I can show you with some spiffy code blocks.

    def forward(self, x, hidden):
        """
        Perform a forward pass of our model on some input and hidden state.
        """
        batch_size=x.size()

see that x.size() there? I had to change it like this:

    def forward(self, x, hidden):
        """
        Perform a forward pass of our model on some input and hidden state.
        """
        batch_size=x.size(0)

This fixed an error I was getting that looked like this:

Traceback (most recent call last):
  File "multiple_output.py", line 244, in <module>
    output, h = net(inputs.type(torch.LongTensor), h)
  File "C:\Users\John\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\nn\modules\module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "multiple_output.py", line 172, in forward
    sig_out=sig_out.view(batch_size, -1)
RuntimeError: shape '[55, 44]' is invalid for input of size 31460

There now, aren’t those nice code blocks? I am glad they are, because I have no idea why I had to make that change. There is one clue within the error: 55 x 44 x 13 just happens to equal 31460. So, one could presume that our new output scheme caused this sig_out thing to multiply in size by a factor of 13. But how does batch_size=x.size(0) address that? There is no factor of 13 in there; x is 55 x 44, and x.size(0) gives us 55. Well, I will have to revisit this topic once I know what I’m talking about. For now, consider it magic fairy dust.

Or maybe not so magic, because the accuracy of this new network is pretty sad: 10%.

On its surface this does not seem all that surprising, since we now have a single network that is trying to accomplish 13 different things instead of 13 different networks each trying to accomplish a single thing. Why wouldn’t it be terrible? There are a lot of reasons why this whole scheme might be terrible, which I will pick apart in future posts. But now it is time to take a break from the mysteries of neural networks, and actually put these things in a UI where they can be tested with live input. For this I am going to attempt to use Django, since that is another tool that all the hip kids can’t live without. This dovetails nicely since all this stuff is in Python anyway.

Oh one last code block… I should probably let you know what my data looks like, in case you want to try this at home. Here’s a sample:

    {
        "sentence": "Here's a sentence in which I am happy",
        "grateful": 0,
        "happy": 1,
        "hopeful": 0,
        "determined": 0,
        "aware": 0,
        "stable": 0,
        "frustrated": 0,
        "overwhelmed": 0,
        "angry": 0,
        "guilty": 0,
        "lonely": 0,
        "scared": 0,
        "sad": 0
    }, {
        "sentence": "Here's a sentence in which I am sad",
        "grateful": 0,
        "happy": 0,
        "hopeful": 0,
        "determined": 0,
        "aware": 0,
        "stable": 0,
        "frustrated": 0,
        "overwhelmed": 0,
        "angry": 0,
        "guilty": 0,
        "lonely": 0,
        "scared": 0,
        "sad": 1
    }

Repeat some multiple of 55 times.