Elasticsearch is one of the most popular Open-Source search engines. In addition to search, it provides a rich set of features like filtering, sorting, and aggregation. If you need to implement search using Elasticsearch in the Laravel application, you can use the official Elasticsearch client for PHP, but there is a much simpler way.
Laravel team developed a unified package to index and search data from Eloquent models using different full-text search engines. It is laravel/scout. The package uses drivers to work with different search engines, including Elasticsearch. In this article, I will show how to use Elasticsearch in Laravel with Scout and Jeroen-g/explorer driver.
Table of Contents
How to Use Elasticsearch in Laravel
Before we start, let’s create an example Laravel application with articles and search functionality by title and content. Scout takes data from Eloquent models and pushes it into the search index. So, we need to create a model which will contain data. You may think that it is better to put all data directly into Elasticsearch without a database. But it is not. Most search engines can’t be considered reliable for long-term data storage. Additionally, you need to re-index all data when you want to change index settings. Therefore, this approach makes sense.
1. Prepare Eloquent Model and Data
Firstly, let’s create migration for model that will create table with name articles:
php artisan make:migration CreateArticlesTable
Insert the following code into the up() method:
…Schema::create("articles", function (Blueprint $table) {
$table->id();
$table->string("title");
$table->text("content");
$table->timestamps();
});
Then, create the model:
php artisan make:model Article
Next, create a factory that makes a few articles with random text:
php artisan make:factory Article
Insert the following code into the factory:
database/factories/ArticleFactory.php<?php
namespace Database\Factories;
use Illuminate\Database\Eloquent\Factories\Factory;
class ArticleFactory extends Factory
{
public function definition()
{
return [
"title" => $this->faker->realText(100),
"content" => $this->faker->realText(200),
];
}
}
Then, create a seeder that uses the factory:
php artisan make:seeder ArticlesSeeder
database/seeders/ArticlesSeeder.php<?php
namespace Database\Seeders;
use Illuminate\Database\Console\Seeds\WithoutModelEvents;
use Illuminate\Database\Seeder;
use App\Models\Article;
class ArticlesSeeder extends Seeder
{
public function run(): void
{
Article::factory()
->count(10)
->create();
}
}
Finally, run migrations and seed database with the sample data:
php artisan migrate
php artisan db:seed --class Database\\Seeders\\ArticlesSeeder
2. Install Scout Package
Run the following command to install Scout:
composer require laravel/scout
Then, publish the Scout configuration file and other package assets:
php artisan vendor:publish --provider="Laravel\Scout\ScoutServiceProvider"
3. Install ElasticSearch Driver
All Scout drivers for Elasticsearch are developed by third-party developers. They often have additional features that can be used only in Elasticsearch:
- synergy/scout-elasticsearch-driver – simple ElasticSearch driver;
- matchish/laravel-scout-elasticsearch – a driver that can search by multiple indexes;
- jeroen-g/explorer – allows using many Elasticsearch features by simple query builder, for example, filtering, conditions, and aggregations.
These drivers implement the same functionality with a few additional opportunities. I prefer jeroen-g/explorer because it has a powerful query builder.
Run this command to install the jeroen-g/explorer package:
composer require jeroen-g/explorer
Then, publish the package configuration:
php artisan vendor:publish --tag=explorer.config
Open the config/scout.php file and change the value of the driver setting to “elastic” or add the following line into the .env file:
.envSCOUT_DRIVER="elastic"
4. Prepare Model
The model which you want to make searchable should implement JeroenG\Explorer\Application\Explored interface and use the Laravel\Scout\Searchable trait. Most methods from the interface are provided by the Searchable trait. You should define only the mappableAs() method. In this method, you can describe the structure and settings for the fields you want to search. It is not always necessary because Elasticsearch can create fields with default settings if no settings are specified so that you can return an empty array. However, if you want to increase performance, provide fields and their types in this method. These settings will be used each time when Scout creates an index. Each field must have type specified, for example:
…[
"title" => [
"type" => "text",
],
];
Also you can specify the following settings:
- analyzer – data analyzer that will split text into tokens on indexing;
- search_analyzer – data analyzer that will split text into tokens on searching;
- fielddata – make text field available for aggregating and sorting;
- fields – nested fields that can contain the same data but utilize different analyzers and settings;
- index – reflect should the field be searchable.
Otherwise, you can use the simplified declaration provided by this driver. For example:
…return [
"title" => "text",
"content" => "text",
];
In this case, you can’t configure nested fields and analyzers, but the code is simpler. Here are some popular field types:
- long
- boolean
- double
- integer
- keyword
- text
- date
If you want to modify global index settings, implement IndexSettings interface and add the indexSettings() method. Here you can configure the number of shards, max result window or create custom filters and analyzers. There are some settings and categories which you can use:
- max_result_window – max number of results for search queries;
- max_ngram_diff – max difference in symbols for ngram filters;
- refresh_interval – determines how often a recent changes will become available;
- analysis – a category with text analyzers and filters definition.
In this article, I will create the Article model with two searchable fields: title and content. Also, I will add a custom analyzer that will split the text into tokens on white spaces:
app/Models/Article.php<?php
namespace App\Models;
use JeroenG\Explorer\Application\Explored;
use JeroenG\Explorer\Application\IndexSettings;
use Laravel\Scout\Searchable;
use Illuminate\Database\Eloquent\Model;
class Article extends Model implements Explored, IndexSettings
{
use Searchable;
//.....
public function indexSettings(): array
{
return [
"analysis" => [
"analyzer" => [
"custom_analyzer" => [
"type" => "custom",
"tokenizer" => "whitespace",
"filter" => ["lowercase"],
],
],
],
];
}
public function mappableAs(): array
{
return [
"title" => [
"type" => "text",
"analyzer" => "custom_analyzer",
"search_analyzer" => "custom_analyzer",
"fields" => [
"keyword" => [
"type" => "keyword",
"ignore_above" => 256,
],
],
],
"content" => [
"type" => "text",
"analyzer" => "custom_analyzer",
"search_analyzer" => "custom_analyzer",
"fields" => [
"keyword" => [
"type" => "keyword",
"ignore_above" => 256,
],
],
],
];
}
}
There are full index settings and field settings declaration. Pay attention to the nested keyword field in the property configuration. It’s the default definition for all text fields. While the main field contains a text split into tokens, the keyword field contains a whole text as one token.
By default, Scout imports all model fields into the index. But in most cases, it is not required. You can override the toSearchableArray() method and provide the data that you want to send into the search index:
app/Models/Article.phppublic function toSearchableArray()
{
return [
"title" => $this->title,
"content" => $this->content,
];
}
5. Configure Driver and Connection
You can configure this Elasticsearch driver in the config/explorer.php file. First, configure the connection to Elasticsearch in the connection section. Specify host, port and scheme here. Also, if your Elasticsearch requires authentication, you can add the auth section. For example:
config/explorer.php "connection" => [
"host" => "localhost",
"port" => "9200",
"scheme" => "https",
"auth" => [
"username" => "elastic",
"password" => "elastic",
],
"ssl" => [
"verify" => false,
]
],
Finally, you should register your models in the indexes section:
config/explorer.php'indexes' => [
\App\Models\Article::class
],
6. Create Index
This Scout has the following commands for index management:
- scout:index – creates an index.
- scout:index-delete – deletes an index.
Elasticsearch can automatically create an index if it does not exist. However, If you want to apply the index settings and mappings configured in the model, you need to create the index manually. Run this command to create articles index:
php artisan scout:index articles
Now, you can open Kibana and ensure that your index was created and has the required settings and properties:
7. Import Data into Index
Scout has a special command for indexing data. It is called scout:import. Let’s import data for the Article model:
php artisan scout:import App\\Models\\Article
The command will display the last id of the added article, and you can switch to Kibana and view how many records appeared in the index. There are ten records in this example:
The advantage of Scout is that all newly-created models will be added to the index. Also, Scout listens to the Update event and updates the index record when the model gets updated. However, Scout can’t track changes in the database, so you have to modify model fields and call the save() method. In addition, Scout will delete the record from the index when you remove the model using the delete() method.
Also, you can write a custom indexer. Just call the searchable() method for the Eloquent query builder. For example, use this code to index all records with non-empty content fields from the articles table:
app/Console/Commands/CustomIndexerCommand.php\App\Models\Article::query()
->whereNotNull("content")
->searchable();
8. Search Data
You can search data in the Elasticsearch index using the static search() method in the model class. For example, let’s find articles with the word “Alice”:
app/Console/Commands/SearchCommand.php$results = \App\Models\Article::search("Alice")->get();
dump($results);
Scout returns an Eloquent collection of models that satisfy the search request. It is very convenient. By default, Elasticsearch will search in all fields available for indexing. If you want to use only a specific field, create a custom query and add it to the search query builder:
app/Console/Commands/SearchCommand.phpuse App\Models\Article;
use JeroenG\Explorer\Domain\Syntax\Matching;
app/Console/Commands/SearchCommand.php$query = Article::search();
$match = new Matching("title", "Alice");
$match->setOperator("and");
$query->must($match);
$results = $query->get();
dump($results);
Note that it is a driver-specific functionality that will not work with other Scout drivers. In the same way, you can build queries with or and/or logic. Use the must() method for queries where all items in the set must be matched and should() method where you want only one of the queries in the set to be matched. Use BoolQuery to group several queries. For example:
app/Console/Commands/SearchCommand.phpuse App\Models\Article;
use JeroenG\Explorer\Domain\Syntax\Matching;
use JeroenG\Explorer\Domain\Syntax\Compound\BoolQuery;
app/Console/Commands/SearchCommand.php$query = Article::search();
$match = new Matching("title", "Alice");
$group = new BoolQuery();
$includeHer = new Matching("title", "her");
$includeThey = new Matching("title", "they");
$group->should($includeHer);
$group->should($includeThey);
$query->must($match);
$query->must($group);
$results = $query->get();
dump($results);
You can add other filters in the same way. Official documentation includes more information about query types and their options. You can found it here.
Wrapping Up
I have explained how to use Elasticsearch with Laravel in this article. As you can see, with the Scout package, it is simple but powerful. You don’t need to write your implementation for indexing models and keep that index updated.
Nice!!! thanks! This manual did the trick for me since the upgrade from elasticsearch 6 to 7 stopped my models syncing with ES.