Transactionfailederror: Too Much Contention on These Datastore Entities. Please Try Again

Are you kidding me, Google?

We utilize Google Cloud. A lot. We use the Kubernetes engine, Datastore, Cloud storage, Deject SQL, Pub/Sub, Cloud DNS etc. For the well-nigh part nosotros love it, but there is one thing that but drives me crazy: the SDKs. The Java ones anyway.

I think the Pub/Sub i is the one that has caused me the most pain (why are you still in Beta by the way?), but today's rant is going to exist mostly almost the Datastore 1.

A lot of our services use Datastore to, well, store data. In general it works but fine, simply sometimes you lot encounter an error. Possibly you get a timeout, or you endeavour to insert something that already exists, or a transaction fails considering of concurrent modifications of an entity.

Whatever the reason, you will get a DatastoreException. It'due south difficult to find documentation on what to expect, so I figured I would just create a piddling Gradle project and endeavour to trigger errors, just to run across what I got. So off I went.

I created a minimal projection, where the only dependency autonomously from Kotlin was the Datastore SDK.

          dependencies {
            compile "org.jetbrains.kotlin:kotlin-stdlib:1.1.2",
            "com.google.cloud:google-cloud-datastore:1.12.0"
}        

So I wrote this little piece of lawmaking, just to see that I could insert an entity:

          @JvmStatic
fun main(args: Array<Cord>) {
            // Zilch special, just connecting to the Datastore emulator
            val datastore = setupDatastore()
            val primal = datastore.newKeyFactory()
            .setKind("examination")
            .newKey("someId")
            val entity = Entity.newBuilder(central)
            .set("message", "Hello globe")
            .build()
            datastore.add(entity)
}        

I ran it.

Ok, so fifty-fifty though the only dependency I have is the Datastore SDK I'm getting conflicts between some transitive dependencies. That's a first for me I remember, I usually need to include at least two dependencies for this to happen..

Ok, so I wanted to have a look at those transitive dependencies.

./gradlew :dependencies — configuration compile | grep protobuf-java
| + — — com.google.protobuf:protobuf-java-util:3.iv.0
| | + — — com.google.protobuf:protobuf-java:three.4.0
| | \ — — com.google.protobuf:protobuf-java:three.3.0 -> 3.iv.0
| + — — com.google.protobuf:protobuf-java:3.3.0 -> 3.4.0
| + — — com.google.protobuf:protobuf-java:3.0.0 -> 3.iv.0
| | | + — — com.google.protobuf:protobuf-java:3.0.0 -> 3.4.0
| | | + — — com.google.protobuf:protobuf-java-util:3.0.0 -> three.4.0 (*)
| | | + — — com.google.protobuf.nano:protobuf-javanano:iii.0.0-blastoff-five
| \ — — com.google.protobuf:protobuf-java:iii.0.0 -> 3.4.0
| \ — — com.google.protobuf:protobuf-java:2.four.1 -> 3.4.0

You guys actually, really like you protobufs.

Ok, so I fixed this by calculation an explicit dependency on protobuf-java-util:3.4.0.

          dependencies {
            compile "org.jetbrains.kotlin:kotlin-stdlib:1.ane.ii",
            "com.google.cloud:google-deject-datastore:ane.12.0",
            "com.google.protobuf:protobuf-coffee-util:iii.iv.0"
}        

Now the entity inserts fine when I run my piece of code. Bang-up. Lets trigger some errors.

Starting time off lets try to add the same entity twice. When doing so, you become a DatastoreException. A DatastoreException has a bunch of fields that are not all that well documented. This is how information technology looks after hitting a breakpoint where I catch the exception.

The message is pretty expert, it tells us what's wrong. Merely you don't really want to use that to, in lawmaking, determine what acquired the error. So lets expect at the other fields. Reason = "INVALID_ARGUMENT", code = iii. Hmm. Code iii is in fact INVALID_ARGUMENT (you need to really look around to discover this, just here information technology is documented).

Looking at the file I linked to to a higher place you can see that:

`INVALID_ARGUMENT` indicates arguments that are problematic regardless of the country of the organization

Well, that'south a poor match for the "entity already exists" error, isn't it? Especially since there is another code, 6, named ALREADY_EXISTS…

So I judge the best affair we can practice if we desire to return say HTTP 409 when we get this exception is to inspect the message. Oh how incorrect that feels.

Moving on. What if we go a timeout?

This one is actually handled quiet well, we go a DatastoreException with code=4 (DEADLINE_EXCEEDED) and reason "DEADLINE_EXCEEDED". Proficient. Just why tin can't I get a DeadlineExceededException instead so I don't need to know well-nigh these magic ints?

What about when a transaction fails because of concurrent modifications?

Y'all get a DatastoreException with code 0, meaning OK, and reason = nix. Not really what you would await, right? Well the cause of the exception is some other DatastoreException, this 1 with code 10 (ABORTED). And the message is "too much contention on these datastore entities. please endeavor again.", which I guess you at least almost can empathise.

Lesser line here is that you lot pretty much need to be an expert to know how to properly handle the exceptions yous get when using the SDK, and/or spend a lot of time trying to trigger them and debugging the code. This could have been made and so much more user friendly by throwing defended exceptions, due east.chiliad. DeadlineExceededException, EntityAlreadyExistsException, ConcurrentModificationException etc.

Ok, rant nearly over. I nevertheless like you Google, but I really hope yous improve your SDKs. And when doing and then, please decouple them from each other so that I'm not forced to used an old Beta version of the Datastore SDK in projects where I apply the Pub/Sub SDK, simply because they are tightly coupled through their common dependencies on 40 other Google libraries..

brodythound.blogspot.com

Source: https://medium.com/@jon_gille/are-you-kidding-me-google-663dacaea9

Share this post