CHECK constraints are enforced from 8.0 and ignored before

MySQL 5.7 parses a CHECK constraint, stores it and never evaluates it, which is worse than rejecting it.

ALTER TABLE orders
  ADD CONSTRAINT chk_status CHECK (status IN
    ('pending','paid','shipped','cancelled'));

-- 5.7: accepted, stored, ignored. every insert succeeds.
-- 8.0: enforced.

-- so the test that matters after an upgrade:
INSERT INTO orders (status) VALUES ('nonsense');
-- ERROR 3819: Check constraint 'chk_status' is violated.

A constraint that exists and does nothing is the worst state, because the schema documents a guarantee the database is not providing. Executing a deliberate violation is the only way to know which behaviour you have, and it is worth doing once after any major version upgrade.