Skip to main content

    Konst: A library for stable hashing

    Here is how load balancing works: To distribute requests among servers using consistent hashing, a proxy takes a hash of part of the request (in our case, the part of the URL that contains the video ID), and uses that hash to choose an available backend server.

    With traditional “modulo hashing”, you simply consider the request hash as a
    very large number. If you take that number modulo the number of available servers, you get the index of the server to use. But this idea is quite limited and it's not realistic in the modern web. The problem is that then number of servers is not constant. A server can fail, a new server might be added when there is more traffic. Traditional hashing approaches "solve" this problem by rehashing (remapping) the data to the new list of servers from scratch. That's slow!

    Then there’s consistent hashing. Consistent hashing uses a more elaborate
    scheme - a hash ring. In a hash ring, each server is assigned multiple hash
    values based on its name or ID, and each request is assigned to the server
    with the “nearest” hash value. The benefit of this added complexity is
    that when a server is added or removed, most requests will map to the same
    server that they did before.So if you have nine servers and add a tenth,
    about 1/10 of requests will have hashes that fall near the newly-added
    server’s hashes, and the other 9/10 will have the same nearest server that
    they did before.

    In the video a modular implementation of such data structure in ES6.
    In addition, it is shown that Konst extends existing stable hashing packages
    in npm since it permits the user of the library to specify his hashing
    algorithms upon initializing without assuming a fixed one as most of npm
    packages currently do.

    Project Members: Konstantinos Pouliasis