Anton Kopylov

dataforseo-ruby: A Client That Respects the Bill

rubyopen-sourceseo

Two of my apps call DataForSEO: ExtensionDash for keyword volumes and Google positions, and the workspace I run my own SEO from. Each had grown its own copy of the HTTP code, and the copies had started to disagree about what an error looks like. So I pulled them out into a gem, dataforseo-ruby.

gem "dataforseo-ruby"

client = DataForSeo::Client.new(login: ENV["DATAFORSEO_LOGIN"], password: ENV["DATAFORSEO_PASSWORD"])

response = client.google_ads_search_volume(keywords: ["reverse image search"], language_code: "en")
response.items  # => [{"keyword" => "reverse image search", "search_volume" => 1500000, ...}]
response.cost   # => 0.09

It has no runtime dependencies: net/http, json and zlib from the standard library, Ruby 3.4 or newer. Most of it is about one fact: every request costs money, including the ones that fail. That fact explains most of the decisions below.

A 200 is not a success

DataForSEO answers almost everything with HTTP 200 and puts the real outcome in a status_code inside the body: once for the request, then once for each task in it. A client that only checks the HTTP status will hand you an empty result as if it were an answer.

So every envelope is checked, top level and per task, and any failure becomes a DataForSeo::Error: AuthenticationError, PaymentRequired, RateLimitError, TransientError, or a plain ApiError carrying the code, message, path and task id. Timeouts, TLS failures and unparseable bodies become ConnectionError. If you send three tasks and get two back, that raises too. A missing task is not an empty one.

Save what you paid for

The bulk endpoints take up to a thousand inputs per request, so the client slices your list and sends it one slice at a time. Pass a block and you get each slice’s response before the next request goes out:

client.bulk_spam_score(targets: domains) do |slice|
  SpamScore.upsert_all(slice.items)   # already paid for; keep it even if the next slice fails
end

Each response carries requested, the inputs sent in that request. That is how a caller tells “the API left this target out” from “we never got to asking”, which are opposite facts and look the same in an empty result.

No retries, on purpose

The client does not retry and does not pace. That sounds lazy, but the reason is that a timeout after the API accepted a request can still be charged, and retrying pays again. Whether a second attempt is worth the money depends on the caller, so the error is the caller’s to handle.

The queue endpoints follow the same rule. Google organic tasks are billed when they are accepted, so google_organic_task_post returns rejected tasks in tasks[] instead of raising. Raising there would throw away the ids of the tasks you have already paid for.

The rest

  • client.post(path, tasks) reaches any other Live endpoint, with the same envelope checks.
  • DataForSeo::Testing::FakeTransport queues bodies, or exceptions, for your own tests and records every call.
  • The README lists the surprises I paid to learn. Google Ads Live allows 12 requests a minute. A keyword over 80 characters or 10 words is rejected. One malformed domain voids a whole bulk_spam_score request, for zero items and zero cost.

It is unofficial, MIT licensed, and on RubyGems. It covers the endpoints I use. If you need another one typed, open an issue or a pull request on GitHub.

← Blog