Learn to Code via Tutorials on Repl.it

← Back to all posts
Creating A Simple Python Socket Server
h
21natzil (832)

Implement Simple Networking Protocols In Python

You want to have your application send information to a server, and back. You also want to keep your code clean. These requirements sound simple, because in essence they are. However without knowledge of certain strategies, many programmers will end up stuck and confused, on seemingly simple problems.

This tutorial will break down my strategy, which I believe to work very well. If you’ve done this sort of programming before, you’ve already developed your own style. If so, you’re welcome to read and comment on what I teach, however the article itself might not bring anything new for you. With that, let’s get started.

We’re going to be using gevent to make our connections asynchronous, as we don’t want to halt all communications when we get a new client. Gevent comes with some built-in servers, which will come in handy. These servers have things called “handlers”. Handlers are callable objects that will receive connections, and as the name states, handle them. Let’s start creating our own handler class.

class Handler:

    def __init__(self, connection, address):
        self.addr = address
        self.conn = connection
        self.start()

    def start(self):
        try:
            self.main()
        finally:
            self.finish()

    def main(self):
        pass

    def finish(self):
        pass

Let’s break down the code. First, we create an init and set the conn and addr properties to the respective arguments. Then we call the start method, which is just shorthand for calling main and finish. The reason we use try and finally is because we want to make sure we clean up our mess, even if an error is raised in main. By using finally instead of except, it’ll make sure that after we clean up, the error will be raised so we can see what went wrong. In this example, we’re creating a server that does math, which will makeup most of main.

import zlib

import msgpack


class Handler:

    ZLIB_SUFFIX = b"\x00\x00\xFF\xFF"

    def __init__(self, connection, address):
        self.addr = address
        self.conn = connection
        self.start()

    def get_msg(self):
        buff = bytearray()
        inflator = zlib.decompressobj()
        while True:
            buff.extend(self.conn.recv(64))
            if not buff.endswith(self.ZLIB_SUFFIX):
                continue
            payload = inflator.decompress(buff) + inflator.flush(self.ZLIB_SUFFIX)
            return msgpack.loads(payload)

    def start(self):
        try:
            self.main()
        finally:
            self.finish()

    def main(self):
        data = self.get_msg()
        if data['equation'] == 'add':
            pass
        else:
            pass

    def finish(self):
        pass

Here comes he most complex part, receiving information. We’re going to use 2 packages to streamline this. The first is zlib, which comes in the standard library. It’s main purpose is to compress and decompress information, which we utilize to make our payloads as small as possible. The second reason we use it is for the ZLIB_SUFFIX. Using the suffix, we know when the message has ended. The second package we use is msgpack. Msgpack is really useful, because it allows us to turn most python datatypes right into bytes, and using less bytes than json normally would. Going back to the code, we can see we added the new get_msg method. It will continue to add bytes to a buffer until the buffer ends with the ZLIB_PREFIX. If it does end with the suffix, we decompress it, and then use msgpack to load it into a dict. (Msgpack can load more than just dicts, however in this case that’s what we’ll be sending / receiving). In the main method, we use the new method to get a dict which will have an equation for the server to do.

import zlib

import msgpack


class Handler:

    ZLIB_SUFFIX = b"\x00\x00\xFF\xFF"

    def __init__(self, connection, address):
        self.addr = address
        self.conn = connection
        self.start()

    def get_msg(self):
        buff = bytearray()
        inflator = zlib.decompressobj()
        while True:
            buff.extend(self.conn.recv(64))
            if not buff.endswith(self.ZLIB_SUFFIX):
                continue
            payload = inflator.decompress(buff) + inflator.flush(self.ZLIB_SUFFIX)
            return msgpack.loads(payload)

    def send(self, answer: int):
        payload = {"answer": answer}
        deflator = zlib.compressobj()
        data = msgpack.dumps(payload)
        data = deflator.compress(data) + deflator.flush(self.ZLIB_SUFFIX)
        self.conn.send(data)

    def start(self):
        try:
            self.main()
        finally:
            self.finish()

    def main(self):
        data = self.get_msg()
        if data['equation'] == 'add':
            self.send(sum(data['numbers']))
        else:
            self.send(
                data['numbers'][0] - data['numbers'][1]
            )

    def finish(self):
        pass

Now we implement the send method to send information back to the client. Inside send is the opposite of get_msg. First we put our answer in a dict, and then use msgpack to turn that into bytes. After that we compress the information using zlib, and send it. We finish the main method, by sending the answer of the math equations to the client.

import zlib

import msgpack
from gevent.server import StreamServer


class Handler:

    ZLIB_SUFFIX = b"\x00\x00\xFF\xFF"

    def __init__(self, connection, address):
        self.addr = address
        self.conn = connection
        self.start()

    def get_msg(self):
        buff = bytearray()
        inflator = zlib.decompressobj()
        while True:
            buff.extend(self.conn.recv(64))
            if not buff.endswith(self.ZLIB_SUFFIX):
                continue
            payload = inflator.decompress(buff) + inflator.flush(self.ZLIB_SUFFIX)
            return msgpack.loads(payload)

    def send(self, answer: int):
        payload = {"answer": answer}
        deflator = zlib.compressobj()
        data = msgpack.dumps(payload)
        data = deflator.compress(data) + deflator.flush(self.ZLIB_SUFFIX)
        self.conn.send(data)

    def start(self):
        try:
            self.main()
        finally:
            self.finish()

    def main(self):
        data = self.get_msg()
        if data['equation'] == 'add':
            self.send(sum(data['numbers']))
        else:
            self.send(
                data['numbers'][0] - data['numbers'][1]
            )

    def finish(self):
        self.conn.close()


if __name__ == '__main__':
    server = StreamServer(('localhost', 12345), Handler)
    server.serve_forever()

Now, we finish the finish method by closing the socket. Then we use our handler class to create a gevent server, which as stated before will automagiclly make this asynchronous, and then run that server forever.

That’s it! You can easily build upon this program to implement whatever server you want. Right now, while our program can’t be exploited by the client very easily, we don’t verify the data the client sends us. If you wanted too, you could use something like Cerberus to confirm the data is what we expected, and possibly catch decompression and msgpack loading errors in case the data was create improperly.

I hope you found this tutorial enlightening, be sure to comment any questions you might have about this design, and I’ll do my best to answer them.

Commentshotnewtop
21natzil (832)

If you enjoyed this and want to support me, you can give some claps on the original medium page