Building a Queue with Vue 3 and Vuex 4
This is a post I’ve been wanting to write for some time. We had to implement a client side queue in two of our apps recently using Vue 3 and Vuex 4.
Now why would you want a client side queue? Well with so much more power given to the web browsers, there are actually times where you might want to perform a long running task in the browser. For example, with FFMPEG WASM you can actually encode videos directly the browser itself, WITHOUT touching a server! For these scenarios, building a queue is the perfect system to make this work.
Since these are still kind of “fringe” scenarios, I wasn’t going to go to the trouble right away by making an officially supported package. However, using Vuex with the reactivity of Vue 3, you can make a simple queue system fairly simply. Let’s get started!
Prerequisites
I’m going to assume you have Vue 3 and Vuex 4 installed as well. We will step through the configuration of these, but I won’t go through everything on how you need to install them.
Base Concepts
The idea of a queue is deferred processing. Essentially meaning, you can place jobs on a queue, they process in order, and you can continue working. When using a queue in Laravel for example, you might want to have the user send a request to an endpoint and return immediately, queuing up the job for later. This allows the user to use your site, get efficient responses, and still have resource intensive processes take place in the background. This will be similar with our client side queue where we want long running processes to run in the background while the user continues to use the app.
When you are using Vuex 4, you are already coming from the mindset that you need to have some “system level state”. What that means is that you need access to data throughout all pages and components. This usually happens within a Single Page Application (NuxtJS or Vue 3) or an InertiaJS app. Any app that doesn’t make a hard refresh between navigation. Or, if you have like one massive page that does a lot of features. An example would be an in browser video editor where it lives on one page and you want the user to export and continue working.
What we will be implementing in this tutorial is a simple queue in Vuex 4. With that queue you can push jobs, see the status, be alerted when the job finishes and view job history. This is great for those long running client side processes. If you want to jump right to the code, check out the repo. Otherwise, let’s get started!
Step 1: Register your store in your Vue 3 app
This is the first step to using Vuex 4 to begin with, registering your store. There are many places you can do this, and will need to find where you configure your app’s set up. If you installed your app using the vue-cli, this will be in your src/main.js.
Ours looks like:
import { createApp } from 'vue'
import { createStore } from 'vuex'
import AppLayout from './App.vue'
/**
* Vuex Queue Set Up
*/
import { queue } from './modules/queue.js';
const store = createStore({
modules: {
queue
}
});
const app = createApp(AppLayout);
app.use(store);
app.mount('#app');There’s a few things I’d like to point out.
First, we need to import the createStore function from vuex. This allows us to build our store and register it with the app.
Second, I already divided up our queue into it’s own module (import { queue } from './modules/queue.js';). This is stored in the /modules directory. I think this is the most beautiful feature of Vuex. The fact you can break up state into small maintainable modules. The queue will have the full gamut of Vuex module functionality, so breaking it up right away allows us to easily manage the queue.
Next, if you installed through the vue-cli or have a similar structure, save what is returned from the createApp() function to a const variable. The vue-cli doesn’t assign this function’s return to anything. We need to do that, because in the next step we need to tell our app to use the store we created:
app.use(store)Finally, make sure you mount the element using app.mount('#app') or whatever the ID of your primary element is. If you installed using vue-cli this happens right after the createApp() method is called. We need to ensure the app uses the store before we mount the element.
You are now ready to start and build out your queue module in Vuex!
Step 2: Build your Queue Vuex 4 Module
As I mentioned in the last step, we created a Queue module. This module can live anywhere you want, but I recommend putting it inside a /modules directory. This way everything is easy to maintain and easy to find.
The first step is to add queue.js to /modules and give it some scaffolding for all of our Vuex functionality. If you are unfamiliar with Vuex, I’d take the time to read the docs before going further. We will be touching on all of the core components.
Our queue.js module should look like:
export const queue = {
namespaced: true,
state: () => ({
}),
actions: {
},
mutations: {
},
getters: {
}
}A quick note, I’d recommend namespacing so none of your modules interfere. Everything will be accessible from the queue/ namespace within our components. Let’s start building out our module.
State
Let’s start with the state. In your queue.js module, add the following to your state:
state: () => ({
pending: [],
completed: [],
active: {}
}),This is the guts of our queue. The pending state will be all of the jobs in the queue. As the queue runs, it will move the first job into active so we can see which job is currently being processed. When the job is finished, the active job will move to the completed array and the next job will begin if it’s available.
Actions
Now we are going to start getting into the weeds. The actions on your queue control how the queue works and moves jobs from pending, to active, to completed. There will be two actions on this queue, but they are fairly dense. Remember, actions can contain logic, but mutations only adjust state within Vuex. Let’s take a look:
actions: {
addJob({ commit, state, dispatch }, job ){
commit( 'addPendingJob', job );
if( Object.keys( state.active ).length == 0 ){
dispatch('startNextJob');
}
},
startNextJob({ commit, state }){
if( Object.keys( state.active ).length > 0 ){
commit( 'addCompletedJob', state.active );
}
if( state.pending.length > 0 ){
commit( 'setActiveJob', state.pending[0] );
commit( 'popCurrentJob' );
}else{
commit( 'setActiveJob', {} );
}
}
},The first action is addJob() which uses the decoupled commit , state , and dispatch parameters as the first local parameter. The second parameter is job. We need the commit function so we can perform mutations on our queue which we will talk about next. Then we need the state parameter to check if we need to start the next job which we will talk about next. The dispatch parameter dispatches the method to start the next job if we have nothing active. Finally, the job is the job object we will add to the queue. I’ll show you the format of that when we dispatch our first queue job.
Right away, we commit the job to pending jobs. We want to make sure that we add it to the queue to keep the order of operations correct. Next, we check to see if we have any active job. If we don’t have an active job, we immediately start the job by dispatching the action startNextJob .
The second action, startNextJob is doesn’t take any special parameters, just the decoupled commit method, and state. First, we check to see if there is an active job. If there is, and we are starting the next job, we add the job to the completed jobs array. Next, we check to see if there is another pending job. If there is another pending job, we set the active job to the next job and then “pop” the current job, meaning we remove the job we just set from pending to active. If there’s no next job, we just set the active job to an empty object.
Those are the only two core actions you need to use when building your queue! Next up, we will go through the mutations and getters, then on to dispatching our first jobs!
Mutations
There are 4 simple mutations in this queue. The power of using Vue 3 with Vuex 4 is the reactivity. A lot of the hard stuff is solved right out of the box! Let’s add the following 4 mutations to our queue:
mutations: {
addPendingJob( state, job ){
state.pending.push( job );
},
setActiveJob( state, job ){
state.active = job;
},
popCurrentJob( state ){
state.pending.shift();
},
addCompletedJob( state, job ){
state.completed.push( job );
}
},Alright, so when we set up state, we set up the pending and completed variables to be arrays. With that, we have a few built in methods that we can use to manipulate these in away to make them act like queue.
First, we have the addPendingJob mutation. This uses the .push() method to push a job on to our pending jobs array.
Second, we have the setActiveJob. This mutation just accepts a job that comes from the startNextJob() action and sets it to the active job in our queue.
Next, we have the popCurrentJob() mutation. Once again, we can use a built in array function to remove the first job in the array, working how a queue data structure would work. We use the .shift() which “pops” the first job from the queue.
Finally, we have the addCompletedJob() mutation. This uses the .push() method again, except we push a job, the previous active job, onto the completed array that’s monitored in our queue.
With those 4 mutations, we have a pretty powerful queue handler right there! Finally, let’s add the getters and we can move to the next steps!
Getters
There are 3 getters to match the 3 pieces of data in our Vuex store. They are:
getters: {
PENDING( state ){
return state.pending;
},
ACTIVE( state ){
return state.active;
},
COMPLETED( state ){
return state.completed;
}
}Each getter works in the Vuex fashion of returning a piece of state. This way we can monitor the queue status from anywhere within our application.
Our entire queue module should look like this:
export const queue = {
namespaced: true,
state: () => ({
pending: [],
completed: [],
active: {}
}),
actions: {
addJob({ commit, state, dispatch }, job ){
commit( 'addPendingJob', job );
if( Object.keys( state.active ).length == 0 ){
dispatch('startNextJob');
}
},
startNextJob({ commit, state }){
if( Object.keys( state.active ).length > 0 ){
commit( 'addCompletedJob', state.active );
}
if( state.pending.length > 0 ){
commit( 'setActiveJob', state.pending[0] );
commit( 'popCurrentJob' );
}else{
commit( 'setActiveJob', {} );
}
}
},
mutations: {
addPendingJob( state, job ){
state.pending.push( job );
},
setActiveJob( state, job ){
state.active = job;
},
popCurrentJob( state ){
state.pending.shift();
},
addCompletedJob( state, job ){
state.completed.push( job );
}
},
getters: {
PENDING( state ){
return state.pending;
},
ACTIVE( state ){
return state.active;
},
COMPLETED( state ){
return state.completed;
}
}
}Now that we have our module completed, let’s put it to work and dispatch some jobs.
Step 3: Dispatching Queue Jobs
Dispatching a job interacts directly with the addJob() action on our queue. That action will handle the movement of our job throughout our queue. Before we dispatch the addJob action, let’s talk about the structure of our job.
The job will be a simple JSON object with two required parameters, an id and a handler() function. The ID will be used so you can visualize whatever is on the queue throughout your app. The handler() bundles the functionality you need to process your job. Before we dive into the handler(), I want to point out, you can add whatever you want to the job object that you need to display or use within your app. This object will be present in the pending, active, and completed sections of your queue.
Let’s take a look at a sample job we can dispatch from anywhere within our app:
this.$store.dispatch('queue/addJob', {
id: 'your-job-id',
handler: function( ){
// Add functionality and data here
// Complete the job. Can also be called from
// a promise like when making an API request
this.$store.dispatch('queue/startNextJob');
}.bind(this)
});What this does is dispatch a job onto the queue using the addJob action. Remember, we namespaced our queue so we have to call queue/addJob to actually add the job. The major point to note is that our handler method has to have .bind(this) at the end. This way we can call the this.$store.dispatch('queue/startNextJob') method. This signals to our queue that the active job is completed and we can work on the next job.
Within the body of the handler() method, you can add whatever you need to process your job. You can include data, API requests, etc. Whenever you are done, just dispatch the startNextJob action and the queue will go to the next job!
With that being said, we need to add 1 more component within our app, and that’s the component to control the queue.
Step 4: Controlling the Queue
Now this component can be very customized. It could even be a mixin that you include on the app level, or through a composition function. For this example, we are just going to throw it in a headless component and include it at the root level of our app. This component simply reads in from the queue and facilitates the processing of the jobs. This is what the component should look like:
import { mapState } from 'vuex';
export default {
computed: {
...mapState('queue', {
pending: 'pending',
active: 'active',
completed: 'completed'
})
},
watch: {
active(){
this.processJob();
}
},
methods: {
processJob(){
if( this.active.handler ){
this.active.handler();
}
}
}
}Since we are operating with vuex, we import the mapState method from vuex. This allows us to view the state right within our component. In the computed section, we mapped the vuex state locally so we can watch for reactive changes. Once again, we namespaced our queue, so we add the namespace as our first parameter.
Next, we have the glue that holds this all together, the watch method that watches our active job. Due to the reactive nature of Vue, whenever there is a change, we can monitor that change. So whenever there is a new active job, we can process it! That’s exactly what happens here:
watch: {
active(){
this.processJob();
}
},When the active job changes, we call the method to process the job.
Next up, we define our only method of processJob(). It’s super straight forward. If there’s a handler on the active job, we call the handler! That decouples the functionality into the job itself so you can have flexible jobs! It also checks to see if the handler is present. This is because if we finish our queue, we reset the active job to an empty object. The reset of the job will trigger the processJob() since the active job changed. We don’t want an error since the handler will not be defined.
That completes the full loop of the queue with Vue (nice rhyme, took me this long to realize that)! Next we will touch on how to display this status anywhere in your app and some optional steps.
Step 5: Viewing Queue Status
So the three pieces of state can be mapped anywhere within your app. All you have to do is make sure the mapState method is imported import { mapState } from 'vuex'; and you map the state locally. From there, you can loop over the pending and completed jobs and/or display the active job and reference keys on that object.
For example, you could show pending jobs like this:
<template>
<div>
<div v-for="( job, index ) in pending v-bind:key="'pending-'+index">
{{ job }}
</div>
</div>
</template>You can also show the active job, and some data like this:
<div>Processing Job {{ active.id }}</div>Those are very simple examples, but that should show a small example of how you can expand in the future!
Optional Steps
There are a few optional features you can add to enhance the functionality of your queue.
Clear Pending Jobs
If you have a lot of tasks and realize you made a mistake, you might need to clear the pending jobs. Adding an action you can call to clear the pending jobs is extremely useful. All you have to do is add the following action:
clearPendingJobs({ commit }){
commit( 'setPendingJobs', [] );
}With the mutation:
setPendingJobs( state, jobs ){
state.pending = jobs;
}Now when you call:
this.$store.dispatch('queue/clearPendingJobs');Your pending jobs will be cleared!
Clear Completed Jobs
Similar to pending jobs, but more likely to occur, is the clearing of the completed jobs. After awhile, these may become irrelevant. Let’s add an action that clears these jobs:
clearCompletedJobs({ commit }){
commit( 'setCompletedJobs', [] );
}Then add the following mutation:
setCompletedJobs( state, jobs ){
state.completed = jobs;
}Now when you call:
this.$store.dispatch('queue/clearCompletedJobs');All of your completed jobs will be cleared.
Remove Individual Job
This is also possible with the queue system. And super useful if you mess up a job and don’t want to re-add everything to make it work. To do this you will need to add the following action:
cancelJob({ commit }, index ){
commit( 'removeJob', index );
}This action takes the index of the job we want to cancel. Next, we need to add the following mutation:
removeJob( state, index ){
state.pending.splice( index, 1 );
}Once again, we can use another array function, splice() provided by Javascript to make this efficient. The first parameter in splice() is the index we wish to remove, while the second parameter is how many elements we wish to remove. We only want to remove 1 element at an index, so that’s all we need.
Now when we want to remove an individual job, we can run the following:
this.$store.dispatch('queue/cancelJob', index );And our job has been removed! Those are just a few of the other helpful queue functions we can add.
Conclusion
Hopefully this helped show you some of the possibilities with Vuex 4 and Vue 3 to create dynamic systems! If you have any requests or other ideas, let me know. I might mess around with Pinia in the future to see how that works as well. If you want to see the final product, check out the repo. It’s very simple, but should demonstrate some of the potential!
If you want to learn more about Single Page Application structuring, check out our book. We are in the process of updating it to Vue 3 & Nuxt 3! We cover a lot more information regarding larger Single Page Applications as well as building an API.
Any questions, feel free to reach out on Twitter or on our community forum!
Become a Sponsor