Welcome to week 3 of trying to write a pathtracer in Lua! If you’ve missed out on week 1 and week 2, definitely check them out first! This week there won’t be any Lua; we will get some rudimentary UI and parallel computing into our little pathtracer, so that hot reloading becomes much more convenient and cool. And although the project is far from finished and usable, it is open sourced now so you guys can take a look!

User Interface

The user interface is, of course, implemented using the awesome ImGui library. The left sidebar can view the images & models Lua created & loaded; and Lua scripts and code can be executed on the right sidebar. The bottom bar indicates the current status of the system - whether it’s running a Lua script or not. It’s very crude right now but I believe the user interface will be fleshed out bit by bit in the future.

When proper multi-threading supported is added for Lua, the bottom bar should show a progress bar to indicate the current progress - at least that’s what I wish. I won’t be going into much detail on designing the UI - if you want to learn about ImGui, the best way is to download the library and read the code in ImGui::ShowDemoWindow() by yourself.

Multithreading

With the introduction of UI, we now have to get basic multithreading support done as well - we don’t want a complex Lua script to lag the whole app. Instead, when a Lua script is running, we want to know about the current status, and what’s being changed. That would be so cool. So first, the application will launch a bunch of worker threads after all other resources are being properly loaded:

int con = std::thread::hardware_concurrency();
if (con == 0)
{
    std::cerr << "WARNING: No hardware hardware concurrency? One thread will be launched regardless to separate Lua from UI." << std::endl;
    con = 1;
}
for (int i = 0; i < con; i++)
{
    // Launch new thread...
    int id = thread_id_counter++;
    threads.push_back(std::thread(worker_thread, std::ref(*this), id));
}

We try to launch as many thread as possible to consume maximum CPU. worker_thread is a static method which constantly waits for new available jobs. And when it comes, it takes the job, executes it, and then wait again.

void App::worker_thread(App &app, int thread_id)
{
    while (app.alive)
    {
        // app.mu is a std::mutex.
        std::unique_lock<std::mutex> lk(app.mu);

        // app.cv is a condition variable.
        app.cv.wait(lk, [&]()
        {
            // if there are available job, or the app is quitting, then aquire lock.
            return !app.jobs.empty() || !app.alive;
        });

        if (!app.alive)
        {
            // Time to kill thyself
            return;
        }

        // Take a job. Any job.
        Job job = app.jobs.front();
        app.jobs.pop();
        lk.unlock();

        switch (job.get_job_type())
        {
            case JobType::Nothing:
                break;

            case JobType::Suicide:
                return;

            case JobType::RunScript:
                // Lua::inst() is the singleton instance of the Lua state machine.
                Lua::inst()->execute_file(job.get_script_path());
                break;

            case JobType::Execute:
                Lua::inst()->execute(job.get_code_injection());
                break;
        }

        {
            std::lock_guard<std::mutex> lk(app.mu);
            app.done_job_count++;
        }
    }
}

A worker waits for a job by first aquiring a lock, then releases that lock to put it under the control of a condition variable. The lock will be aquired again when the job queue’s not empty (!app.jobs.empty()) or the app is quitting to properly terminate itself. Four types of jobs are supported.

1. Doing absolutely nothing.
2. Terminate itself.
3. Run a script.
4. Execute a piece of code.

The Job class is very trivial and is defined as such:

class Job
{
public:
    // Constructors, destructors, getters, setters, etc...

private:
    JobType type;
    std::string script_path;
    std::string code_injection;
};

When the user tries to run some sort of script by pressing on the Execute button, a new job is queued and a single worker is notified:

void App::queue_single_job(const Job &job)
{
    {
        std::lock_guard<std::mutex> lk(mu);
        batch_job_count = 1;
        done_job_count = 0;
        jobs.push(job);
    }
    cv.notify_one();
}

If, in the future, true parallel computing is supported, then our batch_job_count can be wayy larger than 1. The user interface will update accordingly to check if Lua is busy right now.

Image Viewing

Recall what we have done on week 1; when Lua creates an Image, it is conveniently stored in a vector accessible by our C++ code. This makes it very convenient to view the images at runtime. And to do so, we need a little bit of OpenGL. We need to first create a wrapper class ImageGL:

class ImageGL
{
public:
    // Constructors & desctructors
    // ...

    bool import_from_image(const Image &image);
    const std::shared_ptr<Image> get_base_image() const;
    void bind() const;

private:
    bool initialized;
    GLuint texture;
    std::shared_ptr<Image> image;
};

During runtime, the user can select on whatever image their Lua script has created. OpenGL then renders that image by first uploading the image to GPU, and then drawing a textured rectangle.

bool ImageGL::import_from_image(const Image &image)
{
    initialized = false;
    if (texture == GL_NONE)
    {
        glGenTextures(1, &texture);
        glBindTexture(GL_TEXTURE_2D, texture);
        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST);
        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_REPEAT);
        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_REPEAT);
    }

    glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, image.w, image.h, 0, GL_RGBA, GL_UNSIGNED_BYTE, image.get());
    initialized = true;

    return true;
}

The import_from_image is called per-frame to update the image to GPU. Fortunately, we won’t be needing to repeatedly create the texture again, and glTexImage2D is fairly quick. I guess.

This also has one extremely useful property. That is, we can see the final image being generated in real-time.

Conclusion

That’s it for this week. Sorry for the lack of progress - I’ve been kind of lazy this week. But up next, we will definitely implement more and more functionality for our LuaPT.

One big problem I’ve encountered is the Lua state, and its inability to multithread. Multithreading wrecks it up real good and usually ends in a panic. By searching on the Internet I’ve found effil, a multithreading library for Lua - but since we have multithreading implemented ourselves we will probably not use it. We will however use its core idea, achieving multithreading by creating multiple Lua states at once and sync their data - so we have that going for us. As always, stay tuned for more!

Comments

A valid, non-empty username is required.

Comment