Hi Pat,

Hey thanks for your help -- if I run it in that way, it does respond differently, but it still errors out:

-----

** Execute fs:rebuild
Stopped Sphinx
Sent configuration to Sphinx
Index Request has completed:
Sphinx 1.10-beta (r2420)
Copyright (c) 2001-2010, Andrew Aksyonoff
Copyright (c) 2008-2010, Sphinx Technologies Inc (http://sphinxsearch.com)

using config file '/mnt/local/flying-sphinx/f6e2ec1de1da051c0/sphinx.conf'...
indexing index 'article_core'...
ERROR: index 'article_core': sql_connect: could not connect to server: Connection timed out
Is the server running on host "ec2-107-21-110-231.compute-1.amazonaws.com" (10.88.171.212) and accepting
TCP/IP connections on port 5432?
 (DSN=pgsql://fhqohcjqzq:***@ec2-107-21-110-231.compute-1.amazonaws.com:5432/fhqohcjqzq).
total 0 docs, 0 bytes
total 189.432 sec, 0 bytes/sec, 0.00 docs/sec
skipping non-plain index 'article'...
indexing index 'event_core'...
ERROR: index 'event_core': sql_connect: could not connect to server: Connection timed out
Is the server running on host "ec2-107-21-110-231.compute-1.amazonaws.com" (10.88.171.212) and accepting
TCP/IP connections on port 5432?
 (DSN=pgsql://fhqohcjqzq:***@ec2-107-21-110-231.compute-1.amazonaws.com:5432/fhqohcjqzq).
total 0 docs, 0 bytes
total 189.439 sec, 0 bytes/sec, 0.00 docs/sec
skipping non-plain index 'event'...
total 0 reads, 0.000 sec, 0.0 kb/call avg, 0.0 msec/call avg
total 0 writes, 0.000 sec, 0.0 kb/call avg, 0.0 msec/call avg
Started Sphinx

-----

Somewhat confused about those errors. I actually have already filed a ticket for this error log -- here's some relevant information from my Heroku Config:

DATABASE_URL => postgres://fhqohcjqzq:***@ec2-107-21-110-231.compute-1.amazonaws.com/fhqohcjqzq
FLYING_SPHINX_API_KEY => b6b0fb6aba98e6858
FLYING_SPHINX_HOST => ec2-174-129-99-83.compute-1.amazonaws.com
FLYING_SPHINX_IDENTIFIER => f6e2ec1de1da051c0
FLYING_SPHINX_INGRESS => false
FLYING_SPHINX_PORT => 9324

… as you can see, the DATABASE_URL matches what FlyingSphinx is trying to use.

Thank you very much,
Kevin
650 714 2315

Ha ha! Hey thank you. Yes, I was using the shared database there, as it was the staging app.

I've migrated to the new 'dev' dedicated database, and it is indexing now. Apologies that this was somewhat of a stupid question. I'm using dedicated databases on the other two environments of this app and I just didn't think to wonder if that were really the case for this environment as well.

Hey -- if it's not too obnoxious for me to ask you another unrelated question: the CRC32 problem. Now I see:

ERROR: index 'event_core': sql_range_query: ERROR: function crc32(unknown) does not exist
LINE 1: ...me")) as int) AS "date_field", 0 AS "view_count", CRC32('pub...
                                                             ^
I have seen some folks making comments as if there were a solution for this problem, at least with dedicated databases? It would take a significant amount of work for me to make my system work without this, as I have several (many) string columns that I am using to support filtering based on the STI 'type' column, etc.

http://support.flying-sphinx.com/discussions/problems/146-crc32-problem-in-ronin-heroku-database

"The regular thinking-sphinx gem creates the function for you, however it's disabled in flying-sphinx because of the shared database issue."

Thanks,
Kevin

Hi Pat,

Yeah -- this worked. See the migration I used below, will probably be helpful to future folks …

One final problem I am having has to do with a timestamp. It's throwing "sql_range_query: ERROR: integer out of range" when I try to index. I made it work on my development machine by making the sphinx.yml file look like this:

development:
  use_64_bit: true

… however this does not work on the production machine. I've also tried fiddling around with casting the timestamp as a bigint, like this:

has "published_at::abstime::int::bigint", :type => :bigint, :as => 'date_field', :sortable => true

This also doesn't work though, it makes the 'date_field' attribute seemingly disappear.

I think the problem is that my production instance is still running "Sphinx 1.10-beta (r2420)" … I've made my sphinx.yml file look like this:

production:
  version: '2.1.0'
  use_64_bit: true

… but this does not change the version number I see when I run the indexer, it remains "Sphinx 1.10-beta (r2420)."

Do you have any ideas as to why the sphinx.yml file does not seem to be picked up by thinking_sphinx on Heroku? Alternatively, is there any other way to force it to use the 2.x version of Sphinx?

You have been extremely helpful -- thanks again,
Kevin

-----

ps. Future Heroku users may find this database migration to add the postgresql 'crc32' function helpful:

class AddCrc32FunctionToPostgreSql < ActiveRecord::Migration
  def up
    sql = <<-SQL
      CREATE OR REPLACE FUNCTION crc32(word text)
      RETURNS bigint AS $$
        DECLARE tmp bigint;
        DECLARE i int;
        DECLARE j int;
        DECLARE byte_length int;
        DECLARE word_array bytea;
        BEGIN
          IF COALESCE(word, '') = '' THEN
            return 0;
          END IF;

          i = 0;
          tmp = 4294967295;
          byte_length = bit_length(word) / 8;
          word_array = decode(replace(word, E'\\\\', E'\\\\\\\\'), 'escape');
          LOOP
            tmp = (tmp # get_byte(word_array, i))::bigint;
            i = i + 1;
            j = 0;
            LOOP
              tmp = ((tmp >> 1) # (3988292384 * (tmp & 1)))::bigint;
              j = j + 1;
              IF j >= 8 THEN
                EXIT;
              END IF;
            END LOOP;
            IF i >= byte_length THEN
              EXIT;
            END IF;
          END LOOP;
          return (tmp # 4294967295);
        END
      $$ IMMUTABLE LANGUAGE plpgsql;
    SQL

    # execute_raw_sql "CREATE LANGUAGE 'plpgsql';"
    execute_raw_sql sql
  end

  def down
    execute_raw_sql "DROP FUNCTION crc32(word text)"
  end

  def execute_raw_sql sql
    ActiveRecord::Base.connection.execute(sql)
  end
end

I found this bit in your documentation

• sphinx_version: Optional indicator of the Sphinx version you wish to use. Allowed values are 1.10, 2.1.0-beta and 2.0.4

http://flying-sphinx.com/api

So I tried using those values, 2.0.4 specifically, but it makes no difference. I continue to see this warning message output in my logs, despite the presence of a sphinx.yml file:

Sphinx cannot be found on your system. You may need to configure the following
settings in your config/sphinx.yml file:
* bin_path * searchd_binary_name * indexer_binary_name

Sphinx.yml:

production:
port: 9312 version: ' 2.0.4' use_64_bit: true

development:
port: 9312 use_64_bit: true

Thanks,
Kevin

Hi Pat,

Getting back into this today … thank you for your quick response. I'm still unable to influence the version of Sphinx, seemingly. Still unable to get it working correctly in the Heroku environment.

For example, this is what my config/sphinx.yml file looks like:

-----

production:
  port: 9312
  version: ' 2.0.4'
  use_64_bit: true

development:
  port: 9312
  use_64_bit: true

-----

And this is what I see when I run heroku run rake fs:index:

-----

** Execute fs:index
Sent configuration to Sphinx
Index Request has completed:
Sphinx 1.10-beta (r2420)
Copyright (c) 2001-2010, Andrew Aksyonoff
Copyright (c) 2008-2010, Sphinx Technologies Inc (http://sphinxsearch.com)

using config file '/mnt/local/flying-sphinx/5ddedbf219f26f5a4/sphinx.conf'...
indexing index 'article_core'...
ERROR: index 'article_core': sql_range_query: ERROR: integer out of range
 (DSN=pgsql://hondgjssdxorkp:***@ec2-107-22-169-60.compute-1.amazonaws.com:5432/d8bg5j66lpg6u6).
total 0 docs, 0 bytes
total 0.483 sec, 0 bytes/sec, 0.00 docs/sec
skipping non-plain index 'article'...
indexing index 'event_core'...
collected 512 docs, 0.4 MB
sorted 0.1 Mhits, 100.0% done
total 512 docs, 409313 bytes
total 0.556 sec, 735524 bytes/sec, 920.05 docs/sec
skipping non-plain index 'event'...
total 7 reads, 0.001 sec, 58.0 kb/call avg, 0.1 msec/call avg
total 12 writes, 0.001 sec, 65.9 kb/call avg, 0.1 msec/call avg

-----

Note how it reports the Sphinx version as 1.10-beta, and it throws an indexing error 'integer out of range' … I've dealt with this error on my development machine, by using the 2.x version of Sphinx, and also telling it 'use_64_bit' … but I don't seem to be able to get this fix working in my staging environment.

Specifically, this part of the documentation doesn't seem to be working for me at all:

-----

Sphinx Versions (Ruby Only)

You can choose which version of Sphinx you'd like to use through the version setting in your config/sphinx.yml file. If nothing is specified you'll see this warning (but everything will work fine), and it'll default to 1.10-beta:

Sphinx cannot be found on your system. You may need to configure the
following settings in your config/sphinx.yml file:
  * bin_path
  * searchd_binary_name
  * indexer_binary_name

For more information, read the documentation:
http://pat.github.com/ts/en/advanced_config.html

The reason for this is that Heroku doesn't know what version of Sphinx you're using - but we can tell Thinking Sphinx (and Flying Sphinx), and that hides this message. Add (or edit) your config/sphinx.yml file to include the version setting for your production environment:

production:

  
version: '1.10-beta'
If you'd rather use Sphinx 2.1.0-dev instead, just specify that in your config/sphinx.ymlfile:

production:

version: '2.1.0'
Make sure you're using flying-sphinx 0.6.2 or newer if you do want to use Sphinx 2.1.0-dev - earlier versions of the gem don't pass the version details through to Flying Sphinx.

-----

Thanks,
Kevin

Pat -- totally, I've added you as a collaborator.

You should get the email from Heroku shortly. Appreciate your help.

Thanks,
Kevin

Hmm -- hey thank you very much for your help. It's all working great now.

We're subscribed to your Heroku add-on and will stick with it for the foreseeable future.

Also I removed you as a collaborator from that app. Hope you enjoy the rest of your Sunday!

Kevin