By Redis Team
CRUD completed, let's do some searching. In order to search, we need data to search over. Remember that persons folder with all the JSON documents and the load-data.sh shell script? Its time has arrived. Go into that folder and run the script:
cd persons
./load-data.sh
You should get a rather verbose response containing the JSON response from the API and the names of the files you loaded. Like this:
{"entityId":"01FY9Z4RRPKF4K9H78JQ3K3CP3","firstName":"Chris","lastName":"Stapleton","age":43,"verified":true,"location":{"longitude":-84.495,"latitude":38.03},"locationUpdated":"2022-01-01T12:00:00.000Z","skills":["singing","football","coal mining"],"personalStatement":"There are days that I can walk around like I'm alright. And I pretend to wear a smile on my face. And I could keep the pain from comin' out of my eyes. But sometimes, sometimes, sometimes I cry."} <- chris-stapleton.json
{"entityId":"01FY9Z4RS2QQVN4XFYSNPKH6B2","firstName":"David","lastName":"Paich","age":67,"verified":false,"location":{"longitude":-118.25,"latitude":34.05},"locationUpdated":"2022-01-01T12:00:00.000Z","skills":["singing","keyboard","blessing"],"personalStatement":"I seek to cure what's deep inside frightened of this thing that I've become"} <- david-paich.json
{"entityId":"01FY9Z4RSD7SQMSWDFZ6S4M5MJ","firstName":"Ivan","lastName":"Doroschuk","age":64,"verified":true,"location":{"longitude":-88.273,"latitude":40.115},"locationUpdated":"2022-01-01T12:00:00.000Z","skills":["singing","dancing","friendship"],"personalStatement":"We can dance if we want to. We can leave your friends behind. 'Cause your friends don't dance and if they don't dance well they're no friends of mine."} <- ivan-doroschuk.json
{"entityId":"01FY9Z4RSRZFGQ21BMEKYHEVK6","firstName":"Joan","lastName":"Jett","age":63,"verified":false,"location":{"longitude":-75.273,"latitude":40.003},"locationUpdated":"2022-01-01T12:00:00.000Z","skills":["singing","guitar","black eyeliner"],"personalStatement":"I love rock n' roll so put another dime in the jukebox, baby."} <- joan-jett.json
{"entityId":"01FY9Z4RT25ABWYTW6ZG7R79V4","firstName":"Justin","lastName":"Timberlake","age":41,"verified":true,"location":{"longitude":-89.971,"latitude":35.118},"locationUpdated":"2022-01-01T12:00:00.000Z","skills":["singing","dancing","half-time shows"],"personalStatement":"What goes around comes all the way back around."} <- justin-timberlake.json
{"entityId":"01FY9Z4RTD9EKBDS2YN9CRMG1D","firstName":"Kerry","lastName":"Livgren","age":72,"verified":false,"location":{"longitude":-95.689,"latitude":39.056},"locationUpdated":"2022-01-01T12:00:00.000Z","skills":["poetry","philosophy","songwriting","guitar"],"personalStatement":"All we are is dust in the wind."} <- kerry-livgren.json
{"entityId":"01FY9Z4RTR73HZQXK83JP94NWR","firstName":"Marshal","lastName":"Mathers","age":49,"verified":false,"location":{"longitude":-83.046,"latitude":42.331},"locationUpdated":"2022-01-01T12:00:00.000Z","skills":["rapping","songwriting","comics"],"personalStatement":"Look, if you had, one shot, or one opportunity to seize everything you ever wanted, in one moment, would you capture it, or just let it slip?"} <- marshal-mathers.json
{"entityId":"01FY9Z4RV2QHH0Z1GJM5ND15JE","firstName":"Rupert","lastName":"Holmes","age":75,"verified":true,"location":{"longitude":-2.518,"latitude":53.259},"locationUpdated":"2022-01-01T12:00:00.000Z","skills":["singing","songwriting","playwriting"],"personalStatement":"I like piña coladas and taking walks in the rain."} <- rupert-holmes.json
A little messy, but if you don't see this, then it didn't work!
Now that we have some data, let's add another router to hold the search routes we want to add. Create a file named search-router.js in the routers folder and set it up with imports and exports just like we did in person-router.js:
import { Router } from 'express'
import { personRepository } from '../om/person.js'
export const router = Router()
Import the Router into server.js the same way we did for the personRouter:
/* import routers */
import { router as personRouter } from './routers/person-router.js'
import { router as searchRouter } from './routers/search-router.js'
Then add the searchRouter to the Express app:
/* bring in some routers */
app.use('/person', personRouter)
app.use('/persons', searchRouter)
Router bound, we can now add some routes.
We're going to add a plethora of searches to our new Router. But the first will be the easiest as it's just going to return everything. Go ahead and add the following code to search-router.js:
router.get('/all', async (req, res) => {
const persons = await personRepository.search().return.all()
res.send(persons)
})
Here we see how to start and finish a search. Searches start just like CRUD operations start—on a Repository. But instead of calling .createAndSave(), .fetch(), .save(), or .remove(), we call .search(). And unlike all those other methods, .search() doesn't end there. Instead, it allows you to build up a query (which you'll see in the next example) and then resolve it with a call to .return.all().
With this new route in place, go into the Swagger UI and exercise the /persons/all route. You should see all of the folks you added with the shell script as a JSON array.
In the example above, the query is not specified—we didn't build anything up. If you do this, you'll just get everything. Which is what you want sometimes. But not most of the time. It's not really searching if you just return everything. So let's add a route that lets us find persons by their last name. Add the following code:
router.get('/by-last-name/:lastName', async (req, res) => {
const lastName = req.params.lastName
const persons = await personRepository.search()
.where('lastName').equals(lastName).return.all()
res.send(persons)
})
In this route, we're specifying a field we want to filter on and a value that it needs to equal. The field name in the call to .where() is the name of the field specified in our schema. This field was defined as a string, which matters because the type of the field determines the methods that are available query it.
In the case of a string, there's just .equals(), which will query against the value of the entire string. This is aliased as .eq(), .equal(), and .equalTo() for your convenience. You can even add a little more syntactic sugar with calls to .is and .does that really don't do anything but make your code pretty. Like this:
const persons = await personRepository.search().where('lastName').is.equalTo(lastName).return.all()
const persons = await personRepository.search().where('lastName').does.equal(lastName).return.all()
You can also invert the query with a call to .not:
const persons = await personRepository.search().where('lastName').is.not.equalTo(lastName).return.all()
const persons = await personRepository.search().where('lastName').does.not.equal(lastName).return.all()
In all these cases, the call to .return.all() executes the query we build between it and the call to .search(). We can search on other field types as well. Let's add some routes to search on a number and a boolean field:
router.get('/old-enough-to-drink-in-america', async (req, res) => {
const persons = await personRepository.search()
.where('age').gte(21).return.all()
res.send(persons)
})
router.get('/non-verified', async (req, res) => {
const persons = await personRepository.search()
.where('verified').is.not.true().return.all()
res.send(persons)
})
The number field is filtering persons by age where the age is great than or equal to 21. Again, there are aliases and syntactic sugar:
const persons = await personRepository.search().where('age').is.greaterThanOrEqualTo(21).return.all()
But there are also more ways to query:
const persons = await personRepository.search().where('age').eq(21).return.all()
const persons = await personRepository.search().where('age').gt(21).return.all()
const persons = await personRepository.search().where('age').gte(21).return.all()
const persons = await personRepository.search().where('age').lt(21).return.all()
const persons = await personRepository.search().where('age').lte(21).return.all()
const persons = await personRepository.search().where('age').between(21, 65).return.all()
The boolean field is searching for persons by their verification status. It already has some of our syntactic sugar in it. Note that this query will match a missing value or a false value. That's why I specified .not.true(). You can also call .false() on boolean fields as well as all the variations of .equals.
const persons = await personRepository.search().where('verified').true().return.all()
const persons = await personRepository.search().where('verified').false().return.all()
const persons = await personRepository.search().where('verified').equals(true).return.all()
So, we've created a few routes and I haven't told you to test them. Maybe you have anyhow. If so, good for you, you rebel. For the rest of you, why don't you go ahead and test them now with Swagger? And, going forward, just test them when you want. Heck, create some routes of your own using the provided syntax and try those out too. Don't let me tell you how to live your life.
Of course, querying on just one field is never enough. Not a problem, Redis OM can handle .and() and .or() like in this route:
router.get('/verified-drinkers-with-last-name/:lastName', async (req, res) => {
const lastName = req.params.lastName
const persons = await personRepository.search()
.where('verified').is.true()
.and('age').gte(21)
.and('lastName').equals(lastName).return.all()
res.send(persons)
})
Here, I'm just showing the syntax for .and() but, of course, you can also use .or().
If you've defined a field with a type of text in your schema, you can perform full-text searches against it. The way a text field is searched is different from how a string is searched. A string can only be compared with .equals() and must match the entire string. With a text field, you can look for words within the string.
A text field is optimized for human-readable text, like an essay or song lyrics. It's pretty clever. It understands that certain words (like a, an, or the) are common and ignores them. It understands how words are grammatically similar and so if you search for give, it matches gives, given, giving, and gave too. And it ignores punctuation.
Let's add a route that does full-text search against our personalStatement field:
router.get('/with-statement-containing/:text', async (req, res) => {
const text = req.params.text
const persons = await personRepository.search()
.where('personalStatement').matches(text)
.return.all()
res.send(persons)
})
Note the use of the .matches() function. This is the only one that works with text fields. It takes a string that can be one or more words—space-delimited—that you want to query for. Let's try it out. In Swagger, use this route to search for the word "walk". You should get the following results:
[
{
"entityId": "01FYC7CTR027F219455PS76247",
"firstName": "Rupert",
"lastName": "Holmes",
"age": 75,
"verified": true,
"location": {
"longitude": -2.518,
"latitude": 53.259
},
"locationUpdated": "2022-01-01T12:00:00.000Z",
"skills": [
"singing",
"songwriting",
"playwriting"
],
"personalStatement": "I like piña coladas and taking walks in the rain."
},
{
"entityId": "01FYC7CTNBJD9CZKKWPQEZEW14",
"firstName": "Chris",
"lastName": "Stapleton",
"age": 43,
"verified": true,
"location": {
"longitude": -84.495,
"latitude": 38.03
},
"locationUpdated": "2022-01-01T12:00:00.000Z",
"skills": [
"singing",
"football",
"coal mining"
],
"personalStatement": "There are days that I can walk around like I'm alright. And I pretend to wear a smile on my face. And I could keep the pain from comin' out of my eyes. But sometimes, sometimes, sometimes I cry."
}
]
Notice how the word "walk" is matched for Rupert Holmes' personal statement that contains "walks" and matched for Chris Stapleton's that contains "walk". Now search "walk raining". You'll see that this returns Rupert's entry only even though the exact text of neither of these words is found in his personal statement. But they are grammatically related so it matched them. This is called stemming and it's a pretty cool feature of RediSearch that Redis OM exploits.
And if you search for "a rain walk" you'll still match Rupert's entry even though the word "a" is not in the text. Why? Because it's a common word that's not very helpful with searching. These common words are called stop words and this is another cool feature of RediSearch that Redis OM just gets for free.
RediSearch, and therefore Redis OM, both support searching by geographic location. You specify a point in the globe, a radius, and the units for that radius and it'll gleefully return all the entities therein. Let's add a route to do just that:
router.get('/near/:lng,:lat/radius/:radius', async (req, res) => {
const longitude = Number(req.params.lng)
const latitude = Number(req.params.lat)
const radius = Number(req.params.radius)
const persons = await personRepository.search()
.where('location')
.inRadius(circle => circle
.longitude(longitude)
.latitude(latitude)
.radius(radius)
.miles)
.return.all()
res.send(persons)
})
This code looks a little different than the others because the way we define the circle we want to search is done with a function that is passed into the .inRadius method:
circle => circle.longitude(longitude).latitude(latitude).radius(radius).miles
All this function does is accept an instance of a Circle that has been initialized with default values. We override those values by calling various builder methods to define the origin of our search (i.e. the longitude and latitude), the radius, and the units that radius is measured in. Valid units are miles, meters, feet, and kilometers.
Let's try the route out. I know we can find Joan Jett at around longitude -75.0 and latitude 40.0, which is in eastern Pennsylvania. So use those coordinates with a radius of 20 miles. You should receive in response:
[
{
"entityId": "01FYC7CTPKYNXQ98JSTBC37AS1",
"firstName": "Joan",
"lastName": "Jett",
"age": 63,
"verified": false,
"location": {
"longitude": -75.273,
"latitude": 40.003
},
"locationUpdated": "2022-01-01T12:00:00.000Z",
"skills": [
"singing",
"guitar",
"black eyeliner"
],
"personalStatement": "I love rock n' roll so put another dime in the jukebox, baby."
}
]
Try widening the radius and see who else you can find.
