Türker Yıldırım

Elasticsearch 7 removed mapping types

Deep dive10 min read

The cluster held one index called shop with four types in it, which was the documented way to do this in 2015. 6.x allowed one type per index and warned about the rest; 7.0 in April removes them entirely. There is no in-place upgrade for that, and the client library changes at the same time.

The symptom

$ curl -s localhost:9200/shop/_mapping | jq 'keys'
["shop"]
$ curl -s localhost:9200/shop/_mapping | jq '.shop.mappings | keys'
["customer", "order", "product", "review"]

# and the field that explains why this had to go:
$ curl -s localhost:9200/shop/_mapping | jq '.. | .status? // empty'
{ "type": "keyword" }      # order.status
{ "type": "keyword" }      # customer.status — the SAME Lucene field

Two conceptually unrelated fields sharing one mapping because they share a name. That is the actual reason types are being removed: they looked like tables and were never anything of the sort — one index is one Lucene index, and the type was a filter on a hidden field.

Why it happens

The type was presented as a namespace and implemented as a discriminator column. Fields with the same name across types were the same field underneath, which means a mapping conflict between two unrelated entities, and sparse documents — a product with no customer_email still occupies the field slot — that make the index larger and the scoring worse.

The deprecation ran across two major versions, which is generous by any standard, and it still surprised people because the 6.x warning appears in a log nobody reads.

The fix

An index per entity, behind an alias

# before
PUT  /shop/order/91204
GET  /shop/order/_search

# after
PUT  /orders-v1/_doc/91204
GET  /orders/_search              ← the alias, not the index

POST /_aliases
{ "actions": [
  { "add": { "index": "orders-v1",    "alias": "orders" }},
  { "add": { "index": "customers-v1", "alias": "customers" }}
]}

Every read and write goes through the alias from the first day, which is the decision that makes every future mapping change a swap rather than a deploy. It costs nothing at creation time and cannot be retrofitted without the same downtime it exists to avoid.

The application change is mechanical and touches every query, because the type is in the URL. Doing it as a search and replace against a client wrapper rather than against scattered curl-shaped calls is the difference between an afternoon and a week — which is an argument for the wrapper existing before the upgrade rather than after it.

The reindex, with writes still arriving

POST /_reindex?wait_for_completion=false
{
  "source": {
    "index": "shop",
    "type":  "order",
    "size":  1000
  },
  "dest": { "index": "orders-v1" },
  "script": {
    "source": "ctx._source.remove('_type'); ctx._id = ctx._source.id"
  }
}

Running it asynchronously returns a task id and is the only sensible mode for anything above a few hundred thousand documents — the synchronous form times out at the proxy and leaves a reindex running that nobody is watching. GET /_tasks/<id> reports progress and the task can be cancelled.

$ curl -s localhost:9200/_tasks/nodeId:1841 | jq -c '.task.status'
{"total":412088,"created":318400,"batches":319,"version_conflicts":0}

# and then the catch-up pass, for anything written during the copy
POST /_reindex
{ "source": { "index": "shop", "type": "order",
              "query": { "range": { "updated_at": { "gte": "2019-04-14T09:00:00Z" }}}},
  "dest": { "index": "orders-v1", "version_type": "external" }}

version_type: external on the catch-up is what stops it overwriting a newer document with an older one — without it a second pass can undo a write that happened between the two. Dual-writing to both indices for the duration is the alternative and is more code; for a reindex measured in minutes the catch-up pass is simpler and adequate.

The default shard count moved, and nobody reads that line

# 6.x: an index nobody configured had 5 primary shards
# 7.x: it has 1

$ curl -s 'localhost:9200/_cat/indices?v&h=index,pri,rep,docs.count,store.size'
index        pri rep docs.count store.size
orders-v1      1   1     412088      2.1gb
customers-v1   1   1      88014    412.4mb

# the sizing rule that has not changed: 10-50 GB per shard.
# a 2 GB index on five shards was five times the overhead for nothing.

The old default was widely agreed to be wrong, and over-sharding is the most common cause of a slow small cluster — each shard is a Lucene index with its own memory, its own merges and its own file handles. The change applies only to newly created indices, so a cluster upgraded in place keeps its old layout until something is reindexed, which this migration conveniently is.

Shard count cannot be changed after creation, which is why this is worth deciding rather than accepting. For an index that will grow, one primary shard with a plan to split later using the split API is more defensible than five chosen speculatively.

The client library is part of the upgrade

// elasticsearch-php 6.x
$client->index([
    'index' => 'shop',
    'type'  => 'order',
    'id'    => $order->id,
    'body'  => $document,
]);

// 7.x — 'type' is gone, and passing it is an error rather than ignored
$client->index([
    'index' => 'orders',
    'id'    => $order->id,
    'body'  => $document,
]);

// and the response shape changed: hits.total is an object now
// 6.x: $r['hits']['total']          → 412
// 7.x: $r['hits']['total']['value'] → 412  (and 'relation')

The hits.total change is the one that breaks quietly: in 7 it is an object with a value and a relation, and the relation can be gte rather than eq because totals above ten thousand are no longer counted exactly by default. Code that reads the old shape gets an array where it expected an integer, and code that displays “412 results” may be displaying “at least 10,000”.

track_total_hits: true restores exact counting at the cost it was always paying. Deciding whether the exact number matters is worth doing per query rather than globally — a results page usually does not care, and a report does.

Verifying it worked

$ curl -s localhost:9200/shop/order/_count | jq -r .count
412088
$ curl -s localhost:9200/orders/_count | jq -r .count
412088

# and the queries, replayed from a recorded set
$ ./compare-search-results.sh queries.txt
  188 queries, 188 identical result sets, 0 score differences

$ curl -s 'localhost:9200/_cat/indices?v' | awk '{print $1, $8}'
# total store size: 2.4 GB → 1.9 GB

Replaying a recorded set of real queries against both clusters and diffing the result sets is the only verification that means anything — counts matching says the documents are there and nothing about whether the mapping produces the same matches. The score comparison is the part that catches a field that was analysed in the old index and is a keyword in the new one, which produces the same documents in a different order.

The storage reduction came from the sparse fields disappearing, which was not the goal and is a reasonable summary of why types were a bad idea.

What this costs

Every client library version is now load-bearing in a way it was not. The 7.x PHP client refuses a type parameter rather than ignoring it, the response shapes changed, and a library pinned to 6.x cannot talk to a 7.x cluster at all for some operations. That makes the upgrade a coordinated release across every service touching the cluster, which for four services is a planning exercise rather than a deploy.

The permanent cost is more indices to manage: four entities that were one index are now four indices with four sets of settings, four aliases and four lifecycle policies. That is more correct and it is more surface. Templates covering them by pattern is what keeps it manageable, and setting those up during the migration rather than afterwards is considerably cheaper.