This change actually had a positive impact on the design of the whole
internals for clustering which is now cleaner and easier to maintain.
It is still far from perfect, but we also have to keep performances in
consideration so we can say that we ended up with a good compromise.
Previously Predis assigned an hash to each command instance which was
computed from its key, now we changed approach and the library caches
the slot assigned to each command. This works for both our client-side
sharding cluster and the upcoming redis-cluster, but the former is the
one that needed most changes.
The PredisCluster aggregate connection now only takes an instance of
StrategyInterface, which in turn wraps the chosen distributor. After
all, in order to be able to calculate the assigned slot for a command
or key, the cluster strategy must have access to the distributor that
manages the distribution of the whole keyspace. Nothing really changes
in terms of configurability as it is still possible to decide which
distributor to use for client-side sharding, it is simply different:
$distributor = new Predis\Cluster\Distribution\KetamaRing();
$strategy = new Predis\Cluster\PredisStrategy($distributor);
$cluster = new Predis\Connection\Aggregate\PredisCluster($strategy);
As for the RedisCluster aggregate connection, the only change is that
the mathematical operation of calculating the assigned slot from a key
has been completely moved inside the cluster strategy instance.
The strategy for redis-cluster does not use external distributors so
trying to StrategyInterface::getDistributor() will throw an exception.
This may change in future releases, but this is not a priority since
redis-cluster relies on a fixed, well-defined distribution mechanism.
This change is possible because, after a few changes in redis-cluster,
our default cluster strategy used for client side sharding and the one
used for redis-cluster turned out to be exactly the same, except for
the hashing function used to calculate distribution.
Actually some checks used to enforce correctness are redundant in the
context of redis-cluster (e.g. the one used to make sure that keys in
requests performing cross-keys operations will hash to the same slot,
which is performed by the server) so we could also add a more dumb and
permissive cluster strategy that relies on checks performed by Redis.
Differently to v0.8, the strategy for client-side sharding now uses
the same rules for extracting hash tags from keys especially when
empty tags are found in the string.
This commit represents a breaking change when the first occurrence of
"{}" is found in a key because it will produce a different hash than
previous versions of Predis, thus leading to a different partitioning.
If you really need to stick with the old behavior, you can subclass
Predis\Cluster\PredisStrategy and override extractKeyTag() using the
old implementation of this method and pass the strategy instance when
initializing the cluster connection via client options.
Multi-keys operations are not allowed even when keys generate the same
hash but this will probably be supported in later betas of Redis which
means we will basically end up reusing the whole strategy used for
client-side sharding.
Ported from the v0.8 branch.
We now have a base test case class for Predis (namely PredisTestCase)
grouping various commonly used utility methods shared by all of the
tests in the suite, greatly improving reusability.
We also changed our wording to indentify this kind of abstraction so
instead of using "scripted commands" (kind of broken English) we now
use "scriptable commands".
One test is currently marked as skipped because it makes
Redis crash when the specified MATCH pattern returns one
or more elements.
See http://redis.io/commands/scan for reference.
One test is currently marked as skipped because it makes
Redis crash when the specified MATCH pattern returns one
or more elements.
See http://redis.io/commands/scan for reference.
A missing "use" directive was preventing the hash strategy to properly
use the specific methods of Predis\Command\ScriptedCommand, falling back
to analyzing the raw arguments array of the command.
This is mainly in response to the longstanding issue #36 in which my
proposed solution was fine in terms of functionalities, but eventually
never made into the repository since it was far from being clean enough
for my taste.
Now developers can optionally pass a callable object when creating the
hashring instance to decide how the distributor should extract the hash
from a node (really a connection instance) to populate the ring:
use Predis\Cluster\Distribution\HashRing;
use Predis\Connection\PredisCluster;
$servers = array(
'tcp://10.0.0.1?alias=node01',
'tcp://10.0.0.2?alias=node02',
);
$options = array(
'nodehash' => function ($connection) {
return $connection->getParameters()->alias;
},
'cluster' => function ($options) {
$replicas = HashRing::DEFAULT_REPLICAS;
$hashring = new HashRing($replicas, $options->nodehash);
$cluster = new PredisCluster($hashring);
return $cluster;
},
);
$client = new Predis\Client($servers, $options);
Both HashRing and KetamaPureRing in the Predis\Cluster\Distribution
namespace support this new approach.
We extract the keys from commands using the second argument of EVAL /
EVALSHA which specifies the number of arguments that must be treated
as keys (used to populate the KEYS table in the Lua script) and then
we check if there is only one key since redis-cluster right now does
not support multi-keys requests.
Our scripted command abstraction is also supported.
We extract the keys from commands using the second argument of EVAL /
EVALSHA which specifies the number of arguments that must be treated
as keys (used to populate the KEYS table in the Lua script) and then
we check if all the keys generate the same hash using the usual method.
Our scripted command abstraction is also supported.
It comes without saying that accessing or setting keys from within the
Lua script is something that might not work as expected, so you should
be careful when using EVAL and EVALSHA in the context of client-side
sharding.
Daniele Alessandri
Michal Humpula
Alexey Kupershtokh
nitper