Original Source: https://smashingmagazine.com/2021/10/building-amazon-product-scraper-nodejs/

Have you ever been in a position where you need to intimately know the market for a particular product? Maybe you’re launching some software and need to know how to price it. Or perhaps you already have your own product on the market and want to see which features to add for a competitive advantage. Or maybe you just want to buy something for yourself and want to make sure you get the best bang for your buck.

All these situations have one thing in common: you need accurate data to make the correct decision. Actually, there’s another thing they share. All scenarios can benefit from the use of a web scraper.

Web scraping is the practice of extracting large amounts of web data through the use of software. So, in essence, it’s a way to automate the tedious process of hitting ‘copy’ and then ‘paste’ 200 times. Of course, a bot can do that in the time it took you to read this sentence, so it’s not only less boring but a lot faster, too.

But the burning question is: why would someone want to scrape Amazon pages?

You’re about to find out! But first of all, I’d like to make something clear right now — while the act of scraping publicly available data is legal, Amazon has some measures to prevent it on their pages. As such, I urge you always to be mindful of the website while scraping, take care not to damage it, and follow ethical guidelines.

Recommended Reading: “The Guide To Ethical Scraping Of Dynamic Websites With Node.js And Puppeteer” by Andreas Altheimer

Why You Should Extract Amazon Product Data

Being the largest online retailer on the planet, it’s safe to say that if you want to buy something, you can probably get it on Amazon. So, it goes without saying just how big of a data treasure trove the website is.

When scraping the web, your primary question should be what to do with all that data. While there are many individual reasons, it boils down to two prominent use cases: optimizing your products and finding the best deals.

Let’s start with the first scenario. Unless you’ve designed a truly innovative new product, the chances are that you can already find something at least similar on Amazon. Scraping those product pages can net you invaluable data such as:

The competitors’ pricing strategy
So, that you can adjust your prices to be competitive and understand how others handle promotional deals;
Customer opinions
To see what your future client base cares about most and how to improve their experience;
Most common features
To see what your competition offers to know which functionalities are crucial and which can be left for later.

In essence, Amazon has everything you need for a deep market and product analysis. You’ll be better prepared to design, launch, and expand your product lineup with that data.

The second scenario can apply to both businesses and regular people. The idea is pretty similar to what I mentioned earlier. You can scrape the prices, features, and reviews of all the products you could choose, and so, you’ll be able to pick the one that offers the most benefits for the lowest price. After all, who doesn’t like a good deal?

Not all products deserve this level of attention to detail, but it can make a massive difference with expensive purchases. Unfortunately, while the benefits are clear, many difficulties go along with scraping Amazon.

The Challenges Of Scraping Amazon Product Data

Not all websites are the same. As a rule of thumb, the more complex and widespread a website is, the harder it is to scrape it. Remember when I said that Amazon was the most prominent e-commerce site? Well, that makes it both extremely popular and reasonably complex.

First off, Amazon knows how scraping bots act, so the website has countermeasures in place. Namely, if the scraper follows a predictable pattern, sending requests at fixed intervals, faster than a human could or with almost identical parameters, Amazon will notice and block the IP. Proxies can solve this problem, but I didn’t need them since we won’t be scraping too many pages in the example.

Next, Amazon deliberately uses varying page structures for their products. That is to say, that if you inspect the pages for different products, there’s a good chance that you’ll find significant differences in their structure and attributes. The reason behind this is quite simple. You need to adapt your scraper’s code for a specific system, and if you use the same script on a new kind of page, you’d have to rewrite parts of it. So, they’re essentially making you work more for the data.

Lastly, Amazon is a vast website. If you want to gather large amounts of data, running the scraping software on your computer might turn out to take way too much time for your needs. This problem is further consolidated by the fact that going too fast will get your scraper blocked. So, if you want loads of data quickly, you’ll need a truly powerful scraper.

Well, that’s enough talk about problems, let’s focus on solutions!

How To Build A Web Scraper For Amazon

To keep things simple, we’ll take a step-by-step approach to writing the code. Feel free to work in parallel with the guide.

Look for the data we need

So, here’s a scenario: I’m moving in a few months to a new place, and I’ll need a couple of new shelves to hold books and magazines. I want to know all my options and get as good of a deal as I can. So, let’s go to the Amazon market, search for “shelves”, and see what we get.

The URL for this search and the page we’ll be scraping is here.

Ok, let’s take stock of what we have here. Just by glancing at the page, we can get a good picture about:

how the shelves look;
what the package includes;
how customers rate them;
their price;
the link to the product;
a suggestion for a cheaper alternative for some of the items.

That’s more than we could ask for!

Get the required tools

Let’s ensure we have all the following tools installed and configured before continuing to the next step.

Chrome
We can download it from here.
VSCode
Follow the instructions on this page to install it on your specific device.
Node.js
Before starting using Axios or Cheerio, we need to install Node.js and the Node Package Manager. The easiest way to install Node.js and NPM is to get one of the installers from the Node.Js official source and run it.

Now, let’s create a new NPM project. Create a new folder for the project and run the following command:

npm init -y

To create the web scraper, we need to install a couple of dependencies in our project:

Cheerio
An open-source library that helps us extract useful information by parsing markup and providing an API for manipulating the resulting data. Cheerio allows us to select tags of an HTML document by using selectors: $(“div”). This specific selector helps us pick all <div> elements on a page. To install Cheerio, please run the following command in the projects’ folder:

npm install cheerio

Axios
A JavaScript library used to make HTTP requests from Node.js.

npm install axios

Inspect the page source

In the following steps, we will learn more about how the information is organized on the page. The idea is to get a better understanding of what we can scrape from our source.

The developer tools help us interactively explore the website’s Document Object Model (DOM). We will use the developer tools in Chrome, but you can use any web browser you’re comfortable with.

Let’s open it by right-clicking anywhere on the page and selecting the “Inspect” option:

This will open up a new window containing the source code of the page. As we have said before, we are looking to scrape every shelf’s information.

As we can see from the screenshot above, the containers that hold all the data have the following classes:

sg-col-4-of-12 s-result-item s-asin sg-col-4-of-16 sg-col sg-col-4-of-20

In the next step, we will use Cheerio to select all the elements containing the data we need.

Fetch the data

After we installed all the dependencies presented above, let’s create a new index.js file and type the following lines of code:

const axios = require(“axios”);
const cheerio = require(“cheerio”);

const fetchShelves = async () => {
try {
const response = await axios.get(‘https://www.amazon.com/s?crid=36QNR0DBY6M7J&k=shelves&ref=glow_cls&refresh=1&sprefix=s%2Caps%2C309’);

const html = response.data;

const $ = cheerio.load(html);

const shelves = [];

$(‘div.sg-col-4-of-12.s-result-item.s-asin.sg-col-4-of-16.sg-col.sg-col-4-of-20’).each((_idx, el) => {
const shelf = $(el)
const title = shelf.find(‘span.a-size-base-plus.a-color-base.a-text-normal’).text()

shelves.push(title)
});

return shelves;
} catch (error) {
throw error;
}
};

fetchShelves().then((shelves) => console.log(shelves));

As we can see, we import the dependencies we need on the first two lines, and then we create a fetchShelves() function that, using Cheerio, gets all the elements containing our products’ information from the page.

It iterates over each of them and pushes it to an empty array to get a better-formatted result.

The fetchShelves() function will only return the product’s title at the moment, so let’s get the rest of the information we need. Please add the following lines of code after the line where we defined the variable title.

const image = shelf.find(‘img.s-image’).attr(‘src’)

const link = shelf.find(‘a.a-link-normal.a-text-normal’).attr(‘href’)

const reviews = shelf.find(‘div.a-section.a-spacing-none.a-spacing-top-micro > div.a-row.a-size-small’).children(‘span’).last().attr(‘aria-label’)

const stars = shelf.find(‘div.a-section.a-spacing-none.a-spacing-top-micro > div > span’).attr(‘aria-label’)

const price = shelf.find(‘span.a-price > span.a-offscreen’).text()

let element = {
title,
image,
link: https://amazon.com${link},
price,
}

if (reviews) {
element.reviews = reviews
}

if (stars) {
element.stars = stars
}

And replace shelves.push(title) with shelves.push(element).

We are now selecting all the information we need and adding it to a new object called element. Every element is then pushed to the shelves array to get a list of objects containing just the data we are looking for.

This is how a shelf object should look like before it is added to our list:

{
title: ‘SUPERJARE Wall Mounted Shelves, Set of 2, Display Ledge, Storage Rack for Room/Kitchen/Office – White’,
image: ‘https://m.media-amazon.com/images/I/61fTtaQNPnL._AC_UL320_.jpg’,
link: ‘https://amazon.com/gp/slredirect/picassoRedirect.html/ref=pa_sp_btf_aps_sr_pg1_1?ie=UTF8&adId=A03078372WABZ8V6NFP9L&url=%2FSUPERJARE-Mounted-Floating-Shelves-Display%2Fdp%2FB07H4NRT36%2Fref%3Dsr_1_59_sspa%3Fcrid%3D36QNR0DBY6M7J%26dchild%3D1%26keywords%3Dshelves%26qid%3D1627970918%26refresh%3D1%26sprefix%3Ds%252Caps%252C309%26sr%3D8-59-spons%26psc%3D1&qualifier=1627970918&id=3373422987100422&widgetName=sp_btf’,
price: ‘$32.99’,
reviews: ‘6,171’,
stars: ‘4.7 out of 5 stars’
}

Format the data

Now that we have managed to fetch the data we need, it’s a good idea to save it as a .CSV file to improve readability. After getting all the data, we will use the fs module provided by Node.js and save a new file called saved-shelves.csv to the project’s folder. Import the fs module at the top of the file and copy or write along the following lines of code:

let csvContent = shelves.map(element => {
return Object.values(element).map(item => “${item}”).join(‘,’)
}).join(“n”)

fs.writeFile(‘saved-shelves.csv’, “Title, Image, Link, Price, Reviews, Stars” + ‘n’ + csvContent, ‘utf8’, function (err) {
if (err) {
console.log(‘Some error occurred – file either not saved or corrupted.’)
} else{
console.log(‘File has been saved!’)
}
})

As we can see, on the first three lines, we format the data we have previously gathered by joining all the values of a shelve object using a comma. Then, using the fs module, we create a file called saved-shelves.csv, add a new row that contains the column headers, add the data we have just formatted and create a callback function that handles the errors.

The result should look something like this:

Bonus Tips!
Scraping Single Page Applications

Dynamic content is becoming the standard nowadays, as websites are more complex than ever before. To provide the best user experience possible, developers must adopt different load mechanisms for dynamic content, making our job a little more complicated. If you don’t know what that means, imagine a browser lacking a graphical user interface. Luckily, there is ✨Puppeteer✨ — the magical Node library that provides a high-level API to control a Chrome instance over the DevTools Protocol. Still, it offers the same functionality as a browser, but it must be controlled programmatically by typing a couple of lines of code. Let’s see how that works.

In the previously created project, install the Puppeteer library by running npm install puppeteer, create a new puppeteer.js file, and copy or write along the following lines of code:

const puppeteer = require(‘puppeteer’)

(async () => {
try {
const chrome = await puppeteer.launch()
const page = await chrome.newPage()
await page.goto(‘https://www.reddit.com/r/Kanye/hot/’)
await page.waitForSelector(‘.rpBJOHq2PR60pnwJlUyP0’, { timeout: 2000 })

const body = await page.evaluate(() => {
return document.querySelector(‘body’).innerHTML
})

console.log(body)

await chrome.close()
} catch (error) {
console.log(error)
}
})()

In the example above, we create a Chrome instance and open up a new browser page that is required to go to this link. In the following line, we tell the headless browser to wait until the element with the class rpBJOHq2PR60pnwJlUyP0 appears on the page. We have also specified how long the browser should wait for the page to load (2000 milliseconds).

Using the evaluate method on the page variable, we instructed Puppeteer to execute the Javascript snippets within the page’s context just after the element was finally loaded. This will allow us to access the page’s HTML content and return the page’s body as the output. We then close the Chrome instance by calling the close method on the chrome variable. The resulted work should consist of all the dynamically generated HTML code. This is how Puppeteer can help us load dynamic HTML content.

If you don’t feel comfortable using Puppeteer, note that there are a couple of alternatives out there, like NightwatchJS, NightmareJS, or CasperJS. They are slightly different, but in the end, the process is pretty similar.

Setting user-agent Headers

user-agent is a request header that tells the website you are visiting about yourself, namely your browser and OS. This is used to optimize the content for your set-up, but websites also use it to identify bots sending tons of requests — even if it changes IPS.

Here’s what a user-agent header looks like:

Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.82 Safari/537.36

In the interest of not being detected and blocked, you should regularly change this header. Take extra care not to send an empty or outdated header since this should never happen for a run-fo-the-mill user, and you’ll stand out.

Rate Limiting

Web scrapers can gather content extremely fast, but you should avoid going at top speed. There are two reasons for this:

Too many requests in short order can slow down the website’s server or even bring it down, causing trouble for the owner and other visitors. It can essentially become a DoS attack.
Without rotating proxies, it’s akin to loudly announcing that you’re using a bot since no human would send hundreds or thousands of requests per second.

The solution is to introduce a delay between your requests, a practice called “rate limiting”. (It’s pretty simple to implement, too!)

In the Puppeteer example provided above, before creating the body variable, we can use the waitForTimeout method provided by Puppeteer to wait a couple of seconds before making another request:

await page.waitForTimeout(3000);

Where ms is the number of seconds you would want to wait.

Also, if we would want to do the same thig for the axios example, we can create a promise that calls the setTimeout() method, in order to help us wait for our desired number of miliseconds:

fetchShelves.then(result => new Promise(resolve => setTimeout(() => resolve(result), 3000)))

In this way, you can avoid putting too much pressure on the targeted server and also, bring a more human approach to web scraping.

Closing Thoughts

And there you have it, a step-by-step guide to creating your own web scraper for Amazon product data! But remember, this was just one situation. If you’d like to scrape a different website, you’ll have to make a few tweaks to get any meaningful results.

Related Reading

If you’d still like to see more web scraping in action, here is some useful reading material for you:

“The Ultimate Guide to Web Scraping with JavaScript and Node.Js,” Robert Sfichi
“Advanced Node.JS Web Scraping with Puppeteer,” Gabriel Cioci
“Python Web Scraping: The Ultimate Guide to Building Your Scraper,” Raluca Penciuc

Original Source: http://feedproxy.google.com/~r/abduzeedo/~3/dFu1ww92b6s/botanical-glassworks-blender-3d

Botanical Glassworks in Blender 3D

abduzeedo1027—21

Calvin Sprague shared a 3D project using Blender, Octane, Adobe Illustrator & Photoshop for Botanical Glassworks. The project is a selection of personal works experimenting with flat illustrated botanical shapes transformed into a three-dimensional space. As a beginner using 3d software, Calving found the process quite challenging but that gave him a new appreciation towards 3d artists and how they work with cameras and lighting.

To see more work follow Calvin on

Behance
Instagram
Website

Original Source: https://smashingmagazine.com/2021/10/real-time-multi-user-game/

As the pandemic lingered, the suddenly-remote team I work with became increasingly foosball-deprived. I thought about how to play foosball in a remote setting, but it was clear that simply reconstructing the rules of foosball on a screen would not be a lot of fun.

What _is_ fun is to kick a ball using toy cars — a realization made as I was playing with my 2-year old kid. The same night I set out to build the first prototype for a game that would become Autowuzzler.

The idea is simple: players steer virtual toy cars in a top-down arena that resembles a foosball table. The first team to score 10 goals wins.

Of course, the idea of using cars to play soccer is not unique, but two main ideas should set Autowuzzler apart: I wanted to reconstruct some of the look and feel of playing on a physical foosball table, and I wanted to make sure it is as easy as possible to invite friends or teammates to a quick casual game.

In this article, I’ll describe the process behind the creation of Autowuzzler, which tools and frameworks I chose, and share a few implementation details and lessons I learned.

First Working (Terrible) Prototype

The first prototype was built using the open-source game engine Phaser.js, mostly for the included physics engine and because I already had some experience with it. The game stage was embedded in a Next.js application, again because I already had a solid understanding of Next.js and wanted to focus mainly on the game.

As the game needs to support multiple players in real-time, I utilized Express as a WebSockets broker. Here is where it becomes tricky, though.

Since the physics calculations were done on the client in the Phaser game, I chose a simple, but obviously flawed logic: The first connected client had the doubtful privilege of doing the physics calculations for all game objects, sending the results to the express server, which in turn broadcasted the updated positions, angles and forces back to the other player’s clients. The other clients would then apply the changes to the game objects.

This led to the situation where the first player got to see the physics happening in real-time (it is happening locally in their browser, after all), while all the other players were lagging behind at least 30 milliseconds (the broadcast rate I chose), or — if the first player’s network connection was slow — considerably worse.

If this sounds like poor architecture to you — you’re absolutely right. However, I accepted this fact in favor of quickly getting something playable to figure out if the game is actually fun to play.

Validate The Idea, Dump The Prototype

As flawed as the implementation was, it was sufficiently playable to invite friends for a first test drive. Feedback was very positive, with the major concern being — not surprisingly — the real-time performance. Other inherent problems included the situation when the first player (remember, the one in charge of everything) left the game — who should take over? At this point there was only one game room, so anyone would join the same game. I was also a bit concerned by the bundle size the Phaser.js library introduced.

It was time to dump the prototype and start with a fresh setup and a clear goal.

Project Setup

Clearly, the “first client rules all” approach needed to be replaced with a solution in which the game state lives on the server. In my research, I came across Colyseus, which sounded like the perfect tool for the job.

For the other main building blocks of the game I chose:

Matter.js as a physics engine instead of Phaser.js because it runs in Node and Autowuzzler does not require a full game framework.
SvelteKit as an application framework instead of Next.js, because it just went into public beta at that time. (Besides: I love working with Svelte.)
Supabase.io for storing user-created game PINs.

Let’s look at those building blocks in more detail.

Synchronized, Centralized Game State With Colyseus

Colyseus is a multiplayer game framework based on Node.js and Express. At its core, it provides:

Synchronizing state across clients in an authoritative fashion;
Efficient real-time communication using WebSockets by sending changed data only;
Multi-room setups;
Client libraries for JavaScript, Unity, Defold Engine, Haxe, Cocos Creator, Construct3;
Lifecycle hooks, e.g. room is created, user joins, user leaves, and more;
Sending messages, either as broadcast messages to all users in the room, or to a single user;
A built-in monitoring panel and load test tool.

Note: The Colyseus docs make it easy to get started with a barebones Colyseus server by providing an npm init script and an examples repository.

Creating A Schema

The main entity of a Colyseus app is the game room, which holds the state for a single room instance and all its game objects. In the case of Autowuzzler, it’s a game session with:

two teams,
a finite amount of players,
one ball.

A schema needs to be defined for all properties of the game objects that should be synchronized across clients. For example, we want the ball to synchronize, and so we need to create a schema for the ball:

class Ball extends Schema {
constructor() {
super();
this.x = 0;
this.y = 0;
this.angle = 0;
this.velocityX = 0;
this.velocityY = 0;
}
}
defineTypes(Ball, {
x: “number”,
y: “number”,
angle: “number”,
velocityX: “number”,
velocityY: “number”
});

In the example above, a new class that extends the schema class provided by Colyseus is created; in the constructor, all properties receive an initial value. The position and movement of the ball is described using the five properties: x, y, angle, velocityX, velocityY. Additionally, we need to specify the types of each property. This example uses JavaScript syntax, but you can also use the slightly more compact TypeScript syntax.

Property types can either be primitive types:

string
boolean
number (as well as more efficient integer and float types)

or complex types:

ArraySchema (similar to Array in JavaScript)
MapSchema (similar to Map in JavaScript)
SetSchema (similar to Set in JavaScript)
CollectionSchema (similar to ArraySchema, but without control over indexes)

The Ball class above has five properties of type number: its coordinates (x, y), its current angle and the velocity vector (velocityX, velocityY).

The schema for players is similar, but includes a few more properties to store the player’s name and team’s number, which need to be supplied when creating a Player instance:

class Player extends Schema {
constructor(teamNumber) {
super();
this.name = “”;
this.x = 0;
this.y = 0;
this.angle = 0;
this.velocityX = 0;
this.velocityY = 0;
this.teamNumber = teamNumber;
}
}
defineTypes(Player, {
name: “string”,
x: “number”,
y: “number”,
angle: “number”,
velocityX: “number”,
velocityY: “number”,
angularVelocity: “number”,
teamNumber: “number”,
});

Finally, the schema for the Autowuzzler Room connects the previously defined classes: One room instance has multiple teams (stored in an ArraySchema). It also contains a single ball, therefore we create a new Ball instance in the RoomSchema’s constructor. Players are stored in a MapSchema for quick retrieval using their IDs.

Now, with all the magic happening on the server, the client only handles the input and draws the state it receives from the server to the screen. With one exception:

Interpolation On The Client

Since we are re-using the same Matter.js physics world on the client, we can improve the experienced performance with a simple trick. Rather than only updating the position of a game object, we also synchronize the velocity of the object. This way, the object keeps on moving on its trajectory even if the next update from the server takes longer than usual. So rather than moving objects in discrete steps from position A to position B, we change their position and make them move in a certain direction.

Lifecycle

The Autowuzzler Room class is where the logic concerned with the different phases of a Colyseus room is handled. Colyseus provides several lifecycle methods:

onCreate: when a new room is created (usually when the first client connects);
onAuth: as an authorization hook to permit or deny entry to the room;
onJoin: when a client connects to the room;
onLeave: when a client disconnects from the room;
onDispose: when the room is discarded.

The Autowuzzler room creates a new instance of the physics world (see section “Physics In A Colyseus App”) as soon as it is created (onCreate) and adds a player to the world when a client connects (onJoin). It then updates the physics world 60 times a second (every 16.6 milliseconds) using the setSimulationInterval method (our main game loop):

// deltaTime is roughly 16.6 milliseconds
this.setSimulationInterval((deltaTime) => this.world.updateWorld(deltaTime));

The physics objects are independent of the Colyseus objects, which leaves us with two permutations of the same game object (like the ball), i.e. an object in the physics world and a Colyseus object that can be synced.

As soon as the physical object changes, its updated properties need to be applied back to the Colyseus object. We can achieve that by listening to Matter.js’ afterUpdate event and setting the values from there:

Events.on(this.engine, “afterUpdate”, () => {
// apply the x position of the physics ball object back to the colyseus ball object
this.state.ball.x = this.physicsWorld.ball.position.x;
// … all other ball properties
// loop over all physics players and apply their properties back to colyseus players objects
})

There’s one more copy of the objects we need to take care of: the game objects in the user-facing game.

Client-Side Application

Now that we have an application on the server that handles the synchronization of the game state for multiple rooms as well as physics calculations, let’s focus on building the website and the actual game interface. The Autowuzzler frontend has the following responsibilities:

enables users to create and share game PINs to access individual rooms;
sends the created game PINs to a Supabase database for persistence;
provides an optional “Join a game” page for players to enter the game PIN;
validates game PINs when a player joins a game;
hosts and renders the actual game on a shareable (i.e. unique) URL;
connects to the Colyseus server and handle state updates;
provides a landing (“marketing”) page.

For the implementation of those tasks, I chose SvelteKit over Next.js for the following reasons:

Why SvelteKit?

I have been wanting to develop another app using Svelte ever since I built neolightsout. When SvelteKit (the official application framework for Svelte) went into public beta, I decided to build Autowuzzler with it and accept any headaches that come with using a fresh beta — the joy of using Svelte clearly makes up for it.

These key features made me choose SvelteKit over Next.js for the actual implementation of the game frontend:

Svelte is a UI framework and a compiler and therefore ships minimal code without a client runtime;
Svelte has an expressive templating language and component system (personal preference);
Svelte includes global stores, transitions and animations out of the box, which means: no decision fatigue choosing a global state management toolkit and an animation library;
Svelte supports scoped CSS in single-file-components;
SvelteKit supports SSR, simple but flexible file-based routing and server-side routes for building an API;
SvelteKit allows for each page to run code on the server, e.g. to fetch data that is used to render the page;
Layouts shared across routes;
SvelteKit can be run in a serverless environment.

Creating And Storing Game PINs

Before a user can start playing the game, they first need to create a game PIN. By sharing the PIN with others, they can all access the same game room.

This is a great use case for SvelteKits server-side endpoints in conjunction with Sveltes onMount function: The endpoint /api/createcode generates a game PIN, stores it in a Supabase.io database and outputs the game PIN as a response. This is response is fetched as soon as the page component of the “create” page is mounted:

Storing Game PINs With Supabase.io

Supabase.io is an open-source alternative to Firebase. Supabase makes it very easy to create a PostgreSQL database and access it either via one of its client libraries or via REST.

For the JavaScript client, we import the createClient function and execute it using the parameters supabase_url and supabase_key we received when creating the database. To store the game PIN that is created on each call to the createcode endpoint, all we need to do is to run this simple insert query:

import { createClient } from ‘@supabase/supabase-js’

const database = createClient(
import.meta.env.VITE_SUPABASE_URL,
import.meta.env.VITE_SUPABASE_KEY
);

const { data, error } = await database
.from(“games”)
.insert([{ code: 123456 }]);

Note: The supabase_url and supabase_key are stored in a .env file. Due to Vite — the build tool at the heart of SvelteKit — it is required to prefix the environment variables with VITE_ to make them accessible in SvelteKit.

Accessing The Game

I wanted to make joining an Autowuzzler game as easy as following a link. Therefore, every game room needed to have its own URL based on the previously created game PIN, e.g. https://autowuzzler.com/play/12345.

In SvelteKit, pages with dynamic route parameters are created by putting the dynamic parts of the route in square brackets when naming the page file: client/src/routes/play/[gamePIN].svelte. The value of the gamePIN parameter will then become available in the page component (see the SvelteKit docs for details). In the play route, we need to connect to the Colyseus server, instantiate the physics world to render to the screen, handle updates to game objects, listen to keyboard input and display other UI like the score, and so on.

Connecting To Colyseus And Updating State

The Colyseus client library enables us to connect a client to a Colyseus server. First, let’s create a new Colyseus.Client by pointing it to the Colyseus server (ws://localhost:2567in development). Then join the room with the name we chose earlier (autowuzzler) and the gamePIN from the route parameter. The gamePIN parameter makes sure the user joins the correct room instance (see “match-making” above).

let client = new Colyseus.Client(“ws://localhost:2567”);
this.room = await client.joinOrCreate(“autowuzzler”, { gamePIN });

Since SvelteKit renders pages on the server initially, we need to make sure that this code only runs on the client after the page is done loading. Again, we use the onMount lifecycle function for that use case. (If you’re familiar with React, onMount is similar to the useEffect hook with an empty dependency array.)

onMount(async () => {
let client = new Colyseus.Client(“ws://localhost:2567”);
this.room = await client.joinOrCreate(“autowuzzler”, { gamePIN });
})

Now that we are connected to the Colyseus game server, we can start to listen to any changes to our game objects.

Here’s an example of how to listen to a player joining the room (onAdd) and receiving consecutive state updates to this player:

this.room.state.players.onAdd = (player, key) => {
console.log(`Player has been added with sessionId: ${key}`);

// add player entity to the game world
this.world.createPlayer(key, player.teamNumber);

// listen for changes to this player
player.onChange = (changes) => {
changes.forEach(({ field, value }) => {
this.world.updatePlayer(key, field, value); // see below
});
};
};

In the updatePlayer method of the physics world, we update the properties one by one because Colyseus’ onChange delivers a set of all changed properties.

Note: This function only runs on the client version of the physics world, as game objects are only manipulated indirectly via the Colyseus server.

updatePlayer(sessionId, field, value) {
// get the player physics object by its sessionId
let player = this.world.players.get(sessionId);
// exit if not found
if (!player) return;
// apply changes to the properties
switch (field) {
case “angle”:
Body.setAngle(player, value);
break;
case “x”:
Body.setPosition(player, { x: value, y: player.position.y });
break;
case “y”:
Body.setPosition(player, { x: player.position.x, y: value });
break;
// set velocityX, velocityY, angularVelocity …
}
}

The same procedure applies to the other game objects (ball and teams): listen to their changes and apply the changed values to the client’s physics world.

So far, no objects are moving because we still need to listen to keyboard input and send it to the server. Instead of directly sending events on every keydown event, we maintain a map of currently pressed keys and send events to the Colyseus server in a 50ms loop. This way, we can support pressing multiple keys at the same time and mitigate the pause that happens after the first and consecutive keydown events when the key stays pressed:

let keys = {};
const keyDown = e => {
keys[e.key] = true;
};
const keyUp = e => {
keys[e.key] = false;
};
document.addEventListener(‘keydown’, keyDown);
document.addEventListener(‘keyup’, keyUp);

let loop = () => {
if (keys[“ArrowLeft”]) {
this.room.send(“move”, { direction: “left” });
}
else if (keys[“ArrowRight”]) {
this.room.send(“move”, { direction: “right” });
}
if (keys[“ArrowUp”]) {
this.room.send(“move”, { direction: “up” });
}
else if (keys[“ArrowDown”]) {
this.room.send(“move”, { direction: “down” });
}
// next iteration
requestAnimationFrame(() => {
setTimeout(loop, 50);
});
}
// start loop
setTimeout(loop, 50);

Now the cycle is complete: listen for keystrokes, send the corresponding commands to the Colyseus server to manipulate the physics world on the server. The Colyseus server then applies the new physical properties to all the game objects and propagates the data back to the client to update the user-facing instance of the game.

Minor Nuisances

In retrospect, two things of the category nobody-told-me-but-someone-should-have come to mind:

A good understanding of how physics engines work is beneficial. I spent a considerable amount of time fine-tuning physics properties and constraints. Even though I built a small game with Phaser.js and Matter.js before, there was a lot of trial-and-error to get objects to move in the way I imagined them to.
Real-time is hard — especially in physics-based games. Minor delays considerably worsen the experience, and while synchronizing state across clients with Colyseus works great, it can’t remove computation and transmission delays.

Gotchas And Caveats With SvelteKit

Since I used SvelteKit when it was fresh out of the beta-oven, there were a few gotchas and caveats I would like to point out:

It took a while to figure out that environment variables need to be prefixed with VITE_ in order to use them in SvelteKit. This is now properly documented in the FAQ.
To use Supabase, I had to add Supabase to both the dependencies and devDependencies lists of package.json. I believe this is no longer the case.
SvelteKits load function runs both on the server and the client!
To enable full hot module replacement (including preserving state), you have to manually add a comment line <!– @hmr:keep-all –> in your page components. See FAQ for more details.

Many other frameworks would have been great fits as well, but I have no regrets about choosing SvelteKit for this project. It enabled me to work on the client application in a very efficient way — mostly because Svelte itself is very expressive and skips a lot of the boilerplate code, but also because Svelte has things like animations, transitions, scoped CSS and global stores baked in. SvelteKit provided all the building blocks I needed (SSR, routing, server routes) and although still in beta, it felt very stable and fast.

Deployment And Hosting

Initially, I hosted the Colyseus (Node) server on a Heroku instance and wasted a lot of time getting WebSockets and CORS working. As it turns out, the performance of a tiny (free) Heroku dyno is not sufficient for a real-time use case. I later migrated the Colyseus app to a small server at Linode. The client-side application is deployed by and hosted on Netlify via SvelteKits adapter-netlify. No surprises here: Netlify just worked great!

Conclusion

Starting out with a really simple prototype to validate the idea helped me a lot in figuring out if the project is worth following and where the technical challenges of the game lay. In the final implementation, Colyseus took care of all the heavy lifting of synchronizing state in real-time across multiple clients, distributed in multiple rooms. It’s impressive how quickly a real-time multi-user application can be built with Colyseus — once you figure out how to properly describe the schema. Colyseus’ built-in monitoring panel helps in troubleshooting any synchronizing issues.

What complicated this setup was the physics layer of the game because it introduced an additional copy of each physics-related game object that needed to be maintained. Storing game PINs in Supabase.io from the SvelteKit app was very straightforward. In hindsight, I could have just used an SQLite database to store the game PINs, but trying out new things is half of the fun when building side projects.

Finally, using SvelteKit for building out the frontend of the game allowed me to move quickly — and with the occasional grin of joy on my face.

Now, go ahead and invite your friends to a round of Autowuzzler!

Further Reading on Smashing Magazine

“Get Started With React By Building A Whac-A-Mole Game,” Jhey Tompkins
“How To Build A Real-Time Multiplayer Virtual Reality Game,” Alvin Wan
“Writing A Multiplayer Text Adventure Engine In Node.js,” Fernando Doglio
“The Future Of Mobile Web Design: Video Game Design And Storytelling,” Suzanne Scacca
“How To Build An Endless Runner Game In Virtual Reality,” Alvin Wan

Original Source: https://www.hongkiat.com/blog/instagram-business-marketing-hacks/

(Guest writer: Jigar Agrawal) Many businesses find it challenging to keep up with fast-changing algorithms on the popular social media site, Instagram. Instagram is nonetheless an incredible…

Visit hongkiat.com for full content.

Original Source: https://www.webdesignerdepot.com/2021/10/fighting-your-corner-assertive-seo-in-2021/

The web industry is beset by competing ideals and goals so that the simplicity of numbers makes sense to us: one is more than zero, two is more than one.

When it comes to any metric, there is an understandable temptation to focus on volume. In some cases, absolute metrics make more sense than others. If your goal is to make money, then $1 is marginally better than $0, and $2 is marginally better than $1.

However, even in ecommerce, some conversions are worth more than others; high-value items or items that open up repeat sales are inherently more valuable in the long term.

SEO (Search Engine Optimization) has traditionally been built around a high-traffic numbers game: if enough people visit your site, then sooner or later, someone will convert. But it is far more effective to attract the right type of visitor, the high-value user that will become a customer or even a brand advocate.

The best content does not guarantee success on Google, and neither does good UX or even Core Web Vitals. Content is no longer king. What works is brand recognition.

SERPs Look Different in 2021+

Traditional SEO strategies would have you pack content with keywords. Use the right keywords, have more keywords than your competitor, and you’ll rank higher. SERPs (Search Engine Results Pages) used to be a league table for keywords.

Unfortunately, it’s simply not that easy any longer, in part because Google has lost its self-confidence.

Even before the recent introduction of dark mode for Google Search, its SERPs had started to look very different. [We tend to focus on Google in these articles because Google is by far the biggest search engine, and whatever direction Google moves in, the industry follows — except for FLoC, that’s going down like a lead balloon on Jupiter.]

Google’s meteoric success has been due to its all-powerful algorithm. Anything you publish online is scrutinized, categorized, and archived by the all-seeing, all-knowing algorithm. We create quality content to appeal to the algorithm. We trust in its fairness, its wisdom…

…all of us except Google, who have seen behind the curtain and found the great and powerful algorithm, may as well be an old man pulling levers and tugging on ropes.

Content Is President

Google has never been coy about the inadequacies of the algorithm. Backlinks have been one of the most significant ranking factors of the algorithm for years because a backlink is a human confirmation of quality. A backlink validates the algorithm’s hypothesis that content is worth linking to.

One hundred words or so of keyword dense text requires less processing and has fewer outliers, and so is relatively simple for an algorithm to assess. And yet content of this kind performs poorly on Google.

The reason is simple: human beings don’t want thin content. We want rich, high-quality content. Thin content is unlikely to be validated by a human.

The key to ranking well is to create content to which many people want to link. Not only does this drive traffic, but it validates the page for Google’s algorithm.

There Can Be Only One

One of the key motivating factors in the recent changes to search has been the evolution of technology.

Siri, Bixby, and all manner of cyber-butler are queueing up to answer your question with a single, authoritative statement. Suddenly, top-ten on Google is a lot less desirable because it’s only the top answer that is returned.

Google, and other search engines, cannot afford to rely on the all-seeing, all-knowing algorithm because the all-powerful algorithm is just an educated guess. It’s a very good educated guess, but it’s an educated guess nonetheless.

Until now, an educated guess was sufficient because if the top result were incorrect, something in the top ten would work. But when it’s a single returned result, what search engines need is certainty.

The Single Source of Truth

As part of the push towards a single, correct answer, Google introduced knowledge panels. These are panels within search results that present Google’s top answer to any given question.

Go ahead and search for “Black Widow” and you’ll see a knowledge panel at the top of the results hierarchy. Many searchers will never get beyond this.

Knowledge panels are controversial because Google is deferring to a third authority on the subject — in the case of Black Widow, Google is deferring to Marvel Studios. If someone at Marvel decided to redefine Black Widow from action-adventure to romantic comedy, Google would respect that [bizarre] decision and update the knowledge panel accordingly.

Whether we approve of the move towards single results, knowledge panels, and whatever else develops in the next few years, it’s a moot point. It’s happening. Most of us don’t have the pull of Marvel Studios. So the question is, how do we adapt to this future and become the authority within our niche.

Making Use of sameAs

One of the most significant developments in recent years has been structured data. Structured data is essentially metadata that tells search engines how to interpret content.

Using structured data, you can specify whether content refers to a product, a person, an organization, or many other possible categories. Structured data allows a search engine to understand the difference between Tom Ford, the designer, Tom Ford, the corporation, and Tom Ford, the perfume.

Most structured data extends the generic “thing”. And thing contains a valuable property: sameAs.

sameAs is used to provide a reference to other channels for the same “thing”. In the case of an organization, that means your Facebook page, your Twitter profile, your YouTube channel, and anything else you can think of.

Implementing sameAs provides corroboration of your brand presence. In effect, it’s backlinking to yourself and providing the type of third-party validation Google needs to promote you up the rankings.

Be a Brand

Google prefers established brands because people are more likely to trust them, and therefore consider the result that Google returned as high-quality. They will, in turn, come back to Google the next time they need something, and Google’s business model survives another day.

Google’s results are skewed towards brands, so the best strategy is to act like a brand.

Brands tend to be highly localized entities that dominate a small sector. There’s no benefit to spreading out keywords in the hope of catching a lot of traffic. Instead, identify the area that you are an expert in, then focus your content there.

Develop a presence on social media, but don’t sign up for every service available unless you have the time to maintain them properly; a suspended or lapsed account doesn’t corroborate your value.

Big Fish, Flexible Pond

There’s a pop-psychology question that asks whether you would prefer to be a big fish in a small pond or a small fish in a big pond. The direction that search is moving the correct answer is “Big fish, small pond”.

The problem with metaphors is that they carry irrelevant limitations with them. In that question, we assume that there is the choice of two ponds, both a fixed size. There is no reason the pond cannot be flexible and grow with you as you increase in size.

What matters from an SEO point of view is that you dominate your niche. You must become the single source of truth, the number one search result. If you find that you aren’t number one, then instead of competing for that top spot, reduce your niche until you are the number one authority in your niche.

Be the single source of truth that Google defers to, and the all-powerful algorithm can clamber into its balloon and float away.

Source

p img {display:inline-block; margin-right:10px;}
.alignleft {float:left;}
p.showcase {clear:both;}
body#browserfriendly p, body#podcast p, div#emailbody p{margin:0;}

The post Fighting Your Corner: Assertive SEO in 2021+ first appeared on Webdesigner Depot.