Smart 404

Set Up the Smart 404 Feature

1. Decide on Provider

• This Feature is powered by either OpenAI or Azure OpenAI.
• Once you've chosen a Provider, you'll need to create an account and get authentication details.
  • When setting things up on the Azure side, ensure you choose either the text-embedding-3-small or text-embedding-3-large model. The Feature will not work with other models.

2. Configure Settings under Tools > ClassifAI > Language Processing > Smart 404 > Settings

• Select the proper Provider in the provider dropdown.
• Enter your authentication details.
• Configure any other settings as desired.

3. ElasticPress configuration

Once the Smart 404 Feature is configured, you can then proceed to get ElasticPress set up to index the data.

If on a standard WordPress installation:

• Install and activate the ElasticPress plugin.
• Set your Elasticsearch URL in the ElasticPress settings (ElasticPress > Settings).
• Go to the ElasticPress > Sync settings page and trigger a sync, ensuring this is set to run a sync from scratch. This will send over the new schema to Elasticsearch and index all content, including creating vector embeddings for each post.

If on a WordPress VIP hosted environment:

• Enable Enterprise Search
• Run the VIP-CLI index command. This sends the new schema to Elasticsearch and indexes all content, including creating vector embeddings for each post. Note you may need to use the --setup flag to ensure the schema is created correctly.

At this point all of your content should be indexed, along with the embeddings data. You'll then need to update your 404 template to display the recommended results.

4. Display the recommended results

The Smart 404 Feature comes with a few helper functions that can be used to display the recommended results on your 404 page:

• Directly display the results using the Classifai\render_smart_404_results() function.
• Get the data and then display it in your own way using the Classifai\get_smart_404_results() function.

You will need to directly integrate these functions into your 404 template where desired. The plugin does not automatically display the results on the 404 page for you.

Both functions support the following arguments. If any argument is not provided, the default value set on the settings page will be used:

• $index (string) - The ElasticPress index to search in. Default is post.
• $num (int) - Maximum number of results to display. Default is 5.
• $num_candidates (int) - Maximum number of results to search over. Default is 5000.
• $rescore (bool) - Whether to run a rescore query or not. Can give better results but often is slower. Default is false.
• $score_function (string) - The vector scoring function to use. Default is cosine. Options are cosine, dot_product, l1_norm and l2_norm.

The Classifai\render_smart_404_results() function also supports the following additional arguments:

• $fallback (bool) - Whether to run a fallback WordPress query if no results are found in Elasticsearch. These results will then be rendered. Default is true.

Examples:

// Render the results.
Classifai\render_smart_404_results(
  [
    'index'          => 'post',
    'num'            => 3,
    'num_candidates' => 1000,
    'rescore'        => true,
    'fallback'       => true,
    'score_function' => 'dot_product',
  ]
);

// Get the results.
$results = Classifai\get_smart_404_results(
  [
    'index'          => 'post',
    'num'            => 10,
    'num_candidates' => 8000,
    'rescore'        => false,
    'score_function' => 'cosine',
  ]
);

ob_start();

// Render the results.
foreach ( $results as $result ) {
?>
  <div>
    <?php if ( has_post_thumbnail( $result->ID ) ) : ?>
      <figure>
        <a href="<?php echo esc_url( get_permalink( $result->ID ) ); ?>">
          <?php echo wp_kses_post( get_the_post_thumbnail( $result->ID ) ); ?>
        </a>
      </figure>
    <?php endif; ?>
    <a href="<?php echo esc_url( get_permalink( $result->ID ) ); ?>">
      <?php echo esc_html( $result->post_title ); ?>
    </a>
  </div>
<?php
}

$output = ob_get_clean();
echo $output;

Local Quickstart

If you want to quickly test things locally, ensure you have Docker installed (Docker Desktop recommended) and then run the following command:

docker run -p 9200:9200 -d --name elasticsearch \
  -e "discovery.type=single-node" \
  -e "xpack.security.enabled=false" \
  -e "xpack.security.http.ssl.enabled=false" \
  -e "xpack.license.self_generated.type=basic" \
  docker.elastic.co/elasticsearch/elasticsearch:7.9.0

This will download, install and start Elasticsearch v7.9.0 to your local machine. You can then access Elasticsearch at http://localhost:9200, which is the same URL you can use to configure ElasticPress with. It is recommended that you change the Content Items per Index Cycle setting in ElasticPress to 20 to ensure indexing doesn't timeout. Also be aware of API rate limits on the OpenAI Embeddings API.