Strange behavior when deleting new records

Dave Fawcett's Avatar

Dave Fawcett

12 May, 2020 04:47 PM

I have a really strange problem, for some reason the index isn't staying in sync with activerecord when I delete something that was freshly added. It's a bit confusing to explain, but let me lay out the context a bit.

Our environment:
Ruby 2.5.7
thinking-sphinx 4.4.1 (with delta indexing enabled and working otherwise fine)
flying-sphinx 2.2.0
ts-delayed-delta 2.1

Everything works just dandy, for updates etc, and even for deletion of older content. However, when we create new records and then delete them (we are using activerecord destroy calls) our activerecord complains about receiving stale ID's from flyingsphinx. It's strange because it only seems to happen when onboarding legitimately new content, not old content (even if we re-add and remove old content). I am wondering if perhaps it is related to delta updates not completing a merge before deletion, possibly causing a race condition of sorts.. but further testing seems to rule that out.. So I would appreciate any guesses or insights anyone can provide. Has me stumped so far..

  1. Support Staff 1 Posted by Pat Allan on 13 May, 2020 04:44 AM

    Pat Allan's Avatar

    Hi Dave,

    I'm going to ponder this one a bit more, nothing comes to mind immediately… but I do wonder if it's related to merges, as that's not a piece of functionality I've ever used myself in my own apps, so I'm less familiar with the edge cases.

    I don't suppose you're able to reproduce the issue locally?



  2. 2 Posted by David on 14 May, 2020 05:33 AM

    David's Avatar

    It doesn't seem to, however I wonder the same about merges. Is there a better way to implement a delta setup without using merges? Currently, I have a scheduled job running the merge and assume it is keeping the delta index small and everything running a-ok... ish ;p

  3. Support Staff 3 Posted by Pat Allan on 21 May, 2020 12:55 PM

    Pat Allan's Avatar

    Some further thoughts about all of this…

    • I've set up a test app locally with deltas and merging, and thus far, haven't been able to reproduce the issue.
    • Merges should indeed keep the delta indices small - after each merge, any delta records have their flag set to false, so the next time a record is edited/added, the index will contain only that record (and then further changes until the next merge).
    • Generally I opt for real-time indices these days instead of SQL-backed indices - this also removes the need for deltas. Is there a reason you've chosen SQL-backed indices? (It's certainly a valid approach in some cases, but I figure it's worth me asking why)
    • You've mentioned you're using destroy calls when deleting objects - just to confirm, this is destroy on a single record, thus the ActiveRecord callback lifecycle is invoked? (Though, now I think destroy on a relation will still instantiate all of those objects and call destroy on each and invoke callbacks, so maybe it's fine either way)

Reply to this discussion

Internal reply

Formatting help / Preview (switch to plain text) No formatting (switch to Markdown)

Attaching KB article:


Attached Files

You can attach files up to 10MB

If you don't have an account yet, we need to confirm you're human and not a machine trying to post spam.

Keyboard shortcuts


? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac