Connecting to big data stores

Follow

As well as connecting to a wide range of relational database management systems, you can also connect to a variety of big data stores. Your Managed Storage instance is included as a big data store (see Using Managed Storage). Your data is retrieved using a SELECT statement, so nothing is written or deleted on your database.

This article contains the following sections:

Adding big data store connections

This section contains the following topics:

Connecting to your database

You can connect to your big data store databases using the Data source Builder.

Note: These steps do not apply to BigQuery or ElasticSearch. For information on BigQuery, see Connecting to BigQuery. For information on ElasticSearch, see Connecting to ElasticSearch.

To connect to a big data store

  1. In the Data sources library (), click the + button to add a new connection.
  2. Select one of the icons below.

  3. Enter your host, port, database, user name, and password.

  4. After you enter your database name, you can retrieve your data. You can select a single table from the Single Table tab.
    Note: If you would like to select multiple tables, you must use the Custom SQL or SQL Designer options (see Selecting multiple tables from a database connection).
  5. When you finish, click Next.
  6. Your data is automatically assigned as either a metric or an attribute. You can use the left and right arrows to edit their placements.
  7. Click Next to select a storage option.
  8. Select Advanced to edit cache and time zone options.
  9. Click Save to finish adding your connection.

If you are experiencing a timeout error when trying to connect, please check the following:

  • Database location: If your database is a local, you can connect using BIME Desktop. If you are using the web interface, you will need to whitelist all domains with *.bime.io.
  • Data limits: Some data sources, such as BigQuery and ElasticSearch, limit the number of rows you can import. If your data exceeds the set number of rows, you could experience a timeout error. To solve this, you can create a view in the Custom SQL option using the GROUP BY clause.
  • IP addresses: If you are using the web interface, you might encounter issues with firewall settings. Please check that you have set the required security settings (see Security settings).
  • Incorrect information: If you are still having trouble connecting, you might have entered incorrect information in the Info step of the Data source Builder.

Security settings

If you are going to access your database in the web interface, your database needs to be read-only and open for the "secure.bimedb.com" DNS (always resolving to 107.23.115.201).

If you connect to a database hosted on Amazon EC2 or an Amazon Redshift cluster, please add the following EC2 Security Group to your security group:

  • Account ID: 910041997824
  • EC2 Security Group Name: proxy-wac

Connecting to BigQuery

If you have an instance saved to a BigQuery database, you can connect to your BigQuery account. Your BigQuery data is limited to 100k rows.
Note: If your dataset is based on a view, you must add an ORDER BY clause to the view in BigQuery.
To connect to BigQuery
  1. In the Data sources library (), click the + button to add a new connection.
  2. Select the BigQuery icon ().
  3. Click Authenticate with Google to authorize your BigQuery account.
  4. Enter your credentials, then click Allow.
  5. Select a project and dataset.

  6. Select your tables to add. You can select a single table form the Single Table tab. If you would like to select multiple tables, you must use Custom SQL or the SQL Designer.
    Note: If you are using the BigQuery wildcard function, it can be hard to determine the schema of the table. To help detect the correct schema, select one table to retrieve schema in the Union tab.

  7. After you select your tables, click Next.
  8. Your data will be automatically assigned as a metric or an attribute. You can use the arrows to edit their placements.
  9. Click the Advanced tab to edit cache and time zone options.
  10. Click Save when you are finished. Your connection will not be established until you click Save.
Have more questions? Submit a request

0 Comments

Please sign in to leave a comment.
Powered by Zendesk