Tingmao: Python, Flask and socket.io/engine.io

In 2020-49 I talked about setting up the test alexa client that’s built with python and flask-socketio. After updating those python dependencies, they allow protocol version 4 now. However, the server doesn’t actually work. Even if I remove anything Alexa specific and just run a very basic server that repeat the same message, when I poll the server with curl, nothing shows up. The python file I used to test this is listed below:

from flask import Flask, send_file
from flask_socketio import SocketIO, emit

app = Flask(__name__)
socketio = SocketIO(app)

def get_alexa_output():
    global socketio
    while True:
        socketio.emit("message", "ping")
        print(".")
        socketio.sleep(0.01)

socketio.start_background_task(target=get_alexa_output)
socketio.run(app)

The expected behaviour of this file is that it will send a message with the text ping to clients in a loop. I used the following curl command to test the socket.io server:

curl 'http://localhost:5000/socket.io/?EIO=4&transport=polling'
curl 'http://localhost:5000/socket.io/?EIO=4&transport=polling&sid=...'

where the ... is replaced with the sid returned by the first command. Second curl call prints nothing for a while, then ends with a 2, which according to the protocol means a ping request. This is probably the server requesting a pong from the client to test the connection state, which is not relevant here.

I also tried to set up this server with a new python environment in docker, and the result is the same, suggesting that the flask-socket.io dependency might itself be broken.

I then tried instead to set up a server in node.js, with the following script:

const engine = require('engine.io');
const server = engine.listen(5000);

server.on('connection', socket => {
  socket.send("hello");
});

Using curl, I can actually see the server sending the message, which means that my testing approach is correct.

Originally this problem only occured to me, but recently the test alexa client stopped working for the two other member and also Brandon as well, likely because they also updated their python dependencies (perhaps inadvertently). The cloud server hosting the skills also used flask and used flask-socketio to send messages to unity clients, but last week we migrated and reinstalled the server due to Brandon running out of free compute credits, and now our unity frontend isn’t able to connect to the cloud server via socket.io as well.

After spending 4 hours trying to fix the python code I was not able to make any progress. Since the problem is occurring for other people as well currently, Brandon is looking into converting everything to just use raw WebSocket.

Victoria: face tracking movements adjustments and setting up the Azure VM

In a meeting with the client last week, I demonstrated the movement with just the head and the client suggested I use a mix of both. For example if there are only slight rotations then only the head moves but if the avatar needs to rotate more, the body will rotate too. I was able to get this working but only through disabling the animator on the avatar game object as I am unsure why the avatar does not rotate its neck through gameObject.transform.Rotate.

I also set up an Azure VM to deploy the Alexa API. For the most part, I was able to follow the user manual from last year’s group but there were a few extra steps needed such as adding an inbound rule.

Kaloyan: Detecting rooms on a floor plan

After thorough research I did manage to find some existing solutions. However, none of these solutions were quite suitable for our case. The ones I found about detecting rooms in an architectural floor plan were too complex to implement in the given time slot (example, example2).

The other option, I came up with, was to detect just the lines and their intersections and from the results, to calculate where the rooms are. However, in this case the solutions I found (like this one and this one) were either not quite accurate (detected a line twice or not detect it at all) or required too much processing power.

As a final solution, I decided to try to develop my own algorithm for this problem.