LucidWorks support Forum » LucidWorks Enterprise

Unable to fetch data or index data through Database connection

(13 posts) (7 voices)
  • Started 2 years ago by alok.ranjan1
  • Latest reply from subscribe

Tags:

  • db
  • DB connection not giving any result
  • empt
  • fetch
  • index
  • record
  • result
  • searc
  1. alok.ranjan1
    Member

    Hi team,

    Please help me out with my query, I have tried connecting to my database in MySql using j-connector JDBC driver provided by mysql.com site and it is not giving me any error but when i try to see the result after creating the DB and giving the right table in the DB it gives no error but niether does it shows any result in "serach" option .

    Please advice if any thing else is needed from my side.

    Posted 2 years ago #
  2. Cassandra Targett

    Could you send us a little bit more information? The log file found at logs/lucid.log may show an error, even if the UI does not. Also, if you could share the SELECT statement with us, and any other details about nested queries or other settings (besides the connection information), that would be helpful.

    Posted 2 years ago #
  3. alexey.serba
    Moderator

    Do you provide id field mapping? LWE Solr XML schema has required "id" field and you need to map some SQL column (or SQL expression) to populate this field.

    http://lucidworks.lucidimagination.com/display/LWEUG/Database+Data+Source#DatabaseDataSource-SQLStatements

    Posted 2 years ago #
  4. alok.ranjan1
    Member

    Hi Cassandra Targett,

    Thanks a lot for replying the query. I am using mysql-connector-java-5.1.7-bin.jar to establish connection with MySQL database.

    JDBC URL is : jdbc:mysql://localhost:3306/crawlerui

    JDBC Driver is : com.mysql.jdbc.Driver and

    Select Statement is : select * from users;

    There is no Incremental indexing and nested query. I just want to index all the collumn of this users table on selection of Search option.

    Is that mandatory to map all table's collumn to index field? If yes, then assist me to map the collumn with field.

    Looking forward for your reply.

    Regards,

    Alok Ranjan

    Posted 2 years ago #
  5. Cassandra Targett

    Alok,

    My colleague pointed this out before you posted your update, but thought I would restate:

    There's a problem in your SQL syntax, which is that you are asking for all the rows from the users table, but you need to specify which field in the table has the primary key and map it to the LWE ID field.  So, if your users table had a primary key column of "user_id" you would create your SELECT statement as so:

    "SELECT user_id AS id, * from users"

    Unfortunately, at this time, LWE will fail silently if this primary key mapping is not explicitly specified.  

    Please let us know if that improves the indexing for you.

    Posted 2 years ago #
  6. alok.ranjan1
    Member

    Hi Cassandra Targett,

    Thanks a ton and pleased to say that i have finally implemented database in lucidworks. I modified the query and get the good result. If, i am not wrong the bottom line is that, for each and every database collumn i need to map with the field either create new field or existing one.

    Now, how to refresh the search menu after deleting the datasource? As i am facing the situation that when i delete any datasource it is still showing in search menu.

    Sometime, the application shown the message like "There has been an application error. Please contact your system administrator". What does it mean by this message? and, how would i resolve this problem?

    Looking forward for you reply.

    Regards,

    Alok Ranjan

    Posted 2 years ago #
  7. charlie
    Member

    We've seen the same message when we were running LWE on a rather overloaded server - I suspected it was a timeout issue.

    Posted 2 years ago #
  8. Mark Miller
    Moderator

    Indeed - I think if certain operations triggered by the UI takes over 60 seconds to complete, you will see a timeout page. This is an area we are aware of and we are working on improvements.

    Posted 2 years ago #
  9. alok.ranjan1
    Member

    Hi Mark,

    Thanks to reply me. When i will increase the timeout period then will that resolve the problem? If yes, then how could I imcrease the timeout period?

    I am using this application into Proxy enabled Intranet, so please, guide me that how could i connect to any URL which is outside from the intranet? When I am accessing the URL of my local Intranet it is giving java.net exception.

    So, Please assist me on accissing the outside url and local intranet url from proxy enabled system.

    Looking forward for your reply.

    Posted 2 years ago #
  10. charlie
    Member

    Mark, here's an idea we've used successfully in the past. Show "There has been an application error.." as usual, but embed in the HTML source more information about the error - so administrators/developers can see more detail, simply by viewing the page source.

    Posted 2 years ago #
  11. aaron.snow
    Member

    Hi Cassandra (and everyone else from Lucid),

    I can't thank you enough for your post about the id field requirement.

    I installed Lucid this morning and am very grateful for the terrific package you've put together. By far the most frustrating part of the entire installation/setup experience so far has been not knowing about the id requirement. As far as I can tell, it's not mentioned anywhere in either the Help or the User Guide, and I struggled for over an hour trying to figure out why no records were being indexed before stumbling onto your forum post.

    Lucid folks, please add this information to the "Database Data Source" and "QuickStart" portions of the User Guide and to the Admin UI Help for configuring database datasources!

     

    Posted 2 years ago #
  12. Cassandra Targett

    Thanks for your feedback - we definitely appreciate it.

    I agree that our documentation could be improved to make this more clear, as we discuss the mapping requirement very generally and without too many specifics.  We'll try to make this change by the next version of our Guide.  Sorry you were frustrated with this initially, but I'm glad you were able to get it working.

    - Cassandra

    Posted 2 years ago #
  13. subscribe
    Member

    Hello

    More or less, I had the same problem here... unfortunately it tooks a while until I recognized that the select statement must contain a alias for "id"; ae.: SELECT customer_idAS id, name FROM foobar.

    But that was not the only thing... searching the index through the UI never returned a hit. After HOURS I found out, that my new created field "name" is not marked fur indexing ;-)! So please check out whenever you create your own fields, that the field options are correct!

    For that, go to ->Index, ->Fields, ->Data Fields. Select here your field, ae. "name", and check the right pane ->Field settings: here you must select options like "Search by Default" and "Include in Results", otherwise the field "name" can't be used for searching ;-).

    PS:
    Same while adding a new field ;-).

    Greetings --- Adrian

    Posted 2 years ago #

RSS feed for this topic