Testing the Server

The easiest way to test the server is to make the client connect to the server and issue client-server messages to trigger specific functions to test. However, this approach may be not feasible if the client build is not ready (or at least the functions to test are not implemented on the client side). In addition, tests like scalability test also face issues because they require massive client connections.

So, this chapter describes how to test the server without the game client.

Approach 1: Leveraging the server to test the server

iFun Engine provides a component that emulates massive client connections.

Note

Encryption, session reliability, and sequence number validation are not supported, yet.

We will use classes in funapi/test/network.h. So, we first need to include the file. The classes in the file are as follows.

funtest::Network and funtest::Session

funtest::Network class is for initialization.

class Network {
 public:
  // Specifies handlers for session open and close.
  // Also specifies the number of threads for network IO.
  // This method must be called before another methods.
  static void Install(const SessionOpenedHandler &opened_handler,
                      const SessionClosedHandler &closed_handler,
                      size_t io_threads_size);

  // Specifies handlers for TCP connection and disconnection.
  static void RegisterTcpTransportHandler(
      const TcpTransportAttachedHandler &tcp_attached_handler,
      const TcpTransportDetachedHandler &tcp_detached_handler);

  // Registers a handler for a client-server message in JSON.
  static void Register(const string &message_type,
                       const MessageHandler &message_handler);

  // Registers a handler for a client-server message in Protobuf.
  static void Register2(const string &message_type,
                        const MessageHandler2 &message_handler);
};

funtest::Session represents a single session.

class Session {
 public:
  DECLARE_CLASS_PTR(Session);

  enum State {
    kOpening,
    kOpened,
    kClosed
  };

  // Creates a new session instance.
  // Note that creating a session instance does not imply a new connection.
  // Connection should be explicitly made via ConnectTCP(...) appeared below.
  static Ptr<Session> Create();

  virtual ~Session();

  // Connects to the server via TCP. If the session hasn't opened a
  // connection before, this will initializes a new one.
  virtual void ConnectTcp(const string &ip, uint16_t port,
                          EncodingScheme encoding) = 0;
  virtual void ConnectTcp(const boost::asio::ip::tcp::endpoint &endpoint,
                          EncodingScheme encoding) = 0;

  // Connects to the server via UDP. If the session hasn't opened a
  // connection before, this will initializes a new one.
  virtual void ConnectUdp(const string &ip, uint16_t port,
                          EncodingScheme encoding) = 0;
  virtual void ConnectUdp(const boost::asio::ip::tcp::endpoint &endpoint,
                          EncodingScheme encoding) = 0;

  // These are not implemented, yet.
  virtual void ConnectHttp(const string &url, EncodingScheme encoding) = 0;
  virtual void ConnectHttp(const string &ip, uint16_t port,
                           EncodingScheme encoding) = 0;
  virtual void ConnectHttp(const boost::asio::ip::tcp::endpoint &endpoint,
                           EncodingScheme encoding) = 0;

  // Returns a session ID.
  virtual const SessionId &id() const = 0;
  // Returns a session state.
  virtual State state() const = 0;

  // Checks whether or not the session is attached to a transport.
  virtual bool IsTransportAttached() const = 0;
  virtual bool IsTransportAttached(TransportProtocol protocol) const = 0;

  // Sends a JSON message to the server.
  virtual void SendMessage(const string &message_type, const Json &message,
                           TransportProtocol protocol) = 0;

  virtual void SendMessage(const string &message_type,
                           const Ptr<FunMessage> &message,
                           TransportProtocol protocol) = 0;

  // Closes an associated transport.
  // The session still remains valid, as a new transport can continue.
  virtual void CloseTransport() = 0;
  virtual void CloseTransport(TransportProtocol protocol) = 0;

  // Closes the session.
  virtual void Close() = 0;

  // You can attach per-session context.
  // The context will be valid throughout the session's lifetime. E.g.)
  //   boost::mutex::scoped_lock lock(*session);  // Explicit locking may be required.
  //   if (session->GetContext()["my_state"] == ...) {
  //     ...
  //     session->GetContext()["my_state"] = ...;
  //   }
  virtual void SetContext(const Json &ctxt) = 0;
  virtual Json &GetContext() = 0;
  virtual const Json &GetContext() const = 0;
  virtual boost::mutex &GetContextMutex() const = 0;
  virtual operator boost::mutex &() const = 0;
};

Example

Code may look like this to emulate a case where there are 300 clients and each of the clients issues 5,000 echo messages.

#include <funapi/test/network.h>

// This is the Start() method in your {project-name}_server.cc.
// Please note that we are adding test-related code inside the method.
static bool Start() {
  // Network initialization.
  funtest::Network::Install(OnSessionOpened, OnSessionClosed, 4);

  // Registers TCP-related handlers
  funtest::Network::RegisterTcpTransportHandler(OnTcpAttached, OnTcpDetached);

  // Registers a handler for the echo message type
  // (echo is one given by default on project creation.)
  funtest::Network::Register2("pbuf_echo", OnEcho);

  // Creates 300 client sessions and makes them connect to the server.
  for (size_t i = 0; i < 300; ++i) {
    // Creates a session instance.
    Ptr<funtest::Session> session = funtest::Session::Create();

    // Remembers the number of messages to send in the session context.
    session->GetContext()["count"] = 5000;

    // We assume that the server is running at 127.0.0.1:8013 and takes Tcp/Protobuf.
    session->ConnectTcp("127.0.0.1", 8013, kProtobufEncoding);
  }

  return true;
}


// Session open handler
void OnSessionOpened(const Ptr<funtest::Session> &session) {
  LOG(INFO) << "[test_client] session created: sid=" << session->id();

  // Randomly generates a text of 5-50 characters.
  string message = RandomGenerator::GenerateAlphanumeric(5, 50);

  // Remembers the generated text to validate a response from the server.
  // We leverages the session context, here.
  session->GetContext()["sent_message"] = message;

  // Creates a protobuf echo message and carries the generated text.
  Ptr<FunMessage> msg(new FunMessage);
  PbufEchoMessage *echo_msg = msg->MutableExtension(pbuf_echo);
  echo_msg->set_msg(message);
  session->SendMessage("pbuf_echo", msg, kTcp);
};

// Session close handler
void OnSessionClosed(const Ptr<funtest::Session> &session, SessionCloseReason reason) {
  LOG(INFO) << "[test_client] session closed: sid=" << session->id();
};

// TCP connection handler
void OnTcpAttached(const Ptr<funtest::Session> &session, bool connected) {
  if (not connected) {
    LOG(ERROR) << "[test_client] failed to connect to the server";
  }
};

// TCP disconnection handler
void OnTcpDetached(const Ptr<funtest::Session> &session) {
  LOG(INFO) << "[test_client] tcp transport disconnected: sid=" << session->id();
};

// Message handler for the echo message type from the server.
void OnEcho(const Ptr<funtest::Session> &session, const Ptr<FunMessage> &msg) {
  BOOST_ASSERT(msg->HasExtension(pbuf_echo));

  // Extracts the echo message type from the response.
  const PbufEchoMessage &echo_msg = msg->GetExtension(pbuf_echo);
  const string &message = echo_msg.msg();

  // Validates the response.
  string sent_message = session->GetContext()["sent_message"].GetString();
  BOOST_ASSERT(sent_message == message);

  // Updates the message count to send.
  Json &count_ctxt = session->GetContext()["count"];
  count_ctxt.SetInteger(count_ctxt.GetInteger() - 1);
  if (count_ctxt.GetInteger() == 0) {
    LOG(INFO) << "[test_client] complete: sid=" << session->id();
    return;
  }

  // Keeps the loop running by randomly generating another text.
  // Please refer to OnSessionOpened().
  {
    string message = RandomGenerator::GenerateAlphanumeric(5, 50);
    session->GetContext()["sent_message"] = message;

    Ptr<FunMessage> msg(new FunMessage);
    PbufEchoMessage *echo_msg = msg->MutableExtension(pbuf_echo);
    echo_msg->set_msg(message);
    session->SendMessage("pbuf_echo", msg, kTcp);
  }
};

Approach 2: Leveraging the client plug-in

Imagine we build a client bot. Bot is usually used to emulate massive client connections. Hence, it’s more desirable to build a bot without extra graphics-related overhead. This can be achieved by leveraging the iFun Engine client plug-in code.

Currently, this article explains how to test the server using C# plug-in for Unity.

Case 1) Using Command Line

You can run the test using C# runtime on command line. Sample code for the tests in C# is in plugin-test/src.

Note

Though the code is based on the iFun Engine’s Unity3D plugin, it does not depends on on the UnityEngine libraries. So, its usage is slightly different than regular iFun Engine plugin. (e.g., how to invoke the Update method)

Please note that we cannot use the regular plug-in code, because it does not use UnityEngine itself.

src/client.cs represents a single game client. Each client bears a FunapiSession instance and supports TCP, UDP, or HTTP. And the client sends/receives echo messages.

class Client
{
    public Client (int id, string server_ip)
    {
        // Client's unique ID.
        id_ = id;

        // server's IP address.
        server_ip_ = server_ip;
    }

    public void Connect (bool session_reliability)
    {
        // Creates a FunapiSession instance and registers callback functions.
        session_ = FunapiSession.Create(server_ip_, session_reliability);
        session_.SessionEventCallback += onSessionEvent;
        session_.TransportEventCallback += onTransportEvent;
        session_.TransportErrorCallback += onTransportError;
        session_.ReceivedMessageCallback += onReceivedMessage;

        // ...

        // Makes a connection to the server.
        ushort port = getPort(protocols[i], encodings[i]);
        session_.Connect(protocols[i], encodings[i], port, option);
    }

    // ...
}

src/main.cs performs the actual test. It creates Client instances, exchanges echo messages, and cleans up the created clients.

class TesterMain
{
    // The number of bot clients
    const int kClientMax = 3;
    // Server's address
    const string kServerIp = "127.0.0.1";

    // A list of client instances.
    List<Client> list_ = new List<Client>();

    // Main method
    public static void Main ()
    {
        // Triggers the test
        new TesterMain().Start();
    }

    void start ()
    {
        // Creates bots.
        for (int i = 0; i < kClientMax; ++i)
        {
            Client client = new Client(i, kServerIp);
            list_.Add(client);
        }

        // Creates a thread in charge of invoking the Update().
        Thread t = new Thread(new ThreadStart(onUpdate));
        t.Start();

        // Tries to Connect
        testConnect();

        // Starts and stops
        testStartStop();

        // Exchanges messages.
        testSendReceive();

        // Terminates the thread.
        t.Abort();
    }

    void onUpdate ()
    {
        // Invokes the Update() once every 33 milliseconds.
        while (true)
        {
            foreach (Client c in list_)
            {
                c.Update();
            }

            Thread.Sleep(33);
        }
    }

    void connect ()
    {
        // Makes each of the clients connect to the server.
        foreach (Client c in list_)
        {
            c.Connect(reliableSession);
        }

        // Waits for the clients to complete connecting.
        while (waitForConnect())
        {
            Thread.Sleep(100);
        }
    }

    ...
}

You can find the whole example in plugin-test/src In case that you re-locate the file or add files, please update plugin-test/makefile.

To build the test code, move into a directory named makefile and run make like this:

# Builds the server.
$ make

# Runs the server.
$ mono tester.exe

Case 2) Using Visual Studio

You may run the tests using client plug-in for Unity and Visual Studio. The sample codes are located in vs2015 directory. We tested the sample codes in Visual Studio 2015.

Project Configuration

When you open the funapi-plugin.sln file in the vs2015 folder, two projects would be loaded. Funapi-plugin is the project that builds the Unity plug-in as a class library, and funapi-plugin-tester project is the unit test project using .NET framework. To run the test cases in Visual Studio, you must add a proeprocessor definition. In the build property of the funapi-plugin project, please add NO_UNITY to the “Conditional compilation symbols.”

Writing a unittest case

There are sample test codes in Tester.cs file in funapi-plugin-tester directory. Function with [TestMethod] attribute is a test functions, and its name will be shonw in the test explorer. You may modify the existing test cases or adding a new one in Tester.cs file. Also, you could define a new test class.

Note

Like the C# runtime testing using command line, you could not use the library from the UnityEngine.

Building and running the unittest

To build the unittest, run “Build Solution” from the “Build” menu. After build completes successfuly, UnitTestExplorer window will be appeared. Alternatively, you can find Test Explorer menu under the Test > Windows > Test Explorer.

You would find the defined test method list on the test explorer. You can run all the test cases sequentialy at once, or you may select and run a single test case.

Logs from the test cases are not displayed imeediately. You can find the test logs after clicking the [Output] under the Result tab.