As of HBase 2.1 methods like HBaseAdmin.available() are deprecated. How do we go about checking if the HBase connection is valid?
It turns out the hbase-client will continually retry the connection transparently.
So this means without changing the retry you might wait indefinitely to get an exception from HBase telling you if (and why) your connection failed.
Here is a snippet that you can incorporate into your existing Java code. Note that this code will not run as-is without being placed into a Java class.
This is a data import handler for Apache Solr. It was originally developed for Dice.com / DHI Inc which they have kindly agreed to opensource.
Tested/Built against Solr 6.3, releases/tags will be added in the future for specific Solr Versions.
Features
many aws authentication options supported
Supports using STS to assume an alternative role, useful for cross-account access.
Automatic aws region setting, if hosted on an ec2 instance, or within the ec2 infrastructure, automatically selects the current region by default.
Supports various form of explicit aws credentials using Java Properties or setting them directly in the DIH configuration.
Support for Dynamo Query Features
key condition expression
filter expression
projection expression
KeyMap
ValueMap
automatically chooses between a Query and a Scan based on the presence of condition expressions.
Configuration
The Dynamo data-import-handler configuration is similar to other DIH configurations, you must use the DataSource with the EntityProcessor as shown in the configuration below.
All configuration parameters support Solr Variable injection.
DIH solrconfig.xml configuration
Witin solrconfig.xml you must include the dynamo data import handler, along with solrs DIH libraries:
Within your solrconfig.xml you must define a request handler, or customize the default one. This is where you will specify the data hanlder configuration you will use with this request handler. Note that when you use the default request handler endpoint of /dataimport you'll be able to use this DIH from the solr admin interface.
Within the root directory of your core/collection configuration you must define a dataimport.properties file. This is where the data-import-handler global settings are stored, but more importantly, each time your data-import-handler runs it will save the last import time for your collection to top of this file.
Scheduling: this file also enables you to schedule the data-import-hanlder to periodically run. You can specify the url parameters to execute each time the dataimport runs.
Here is an example/default dataimport.properties file.
################################################## ## dataimport scheduler properties ## ################################################### to sync or not to sync# 1 - active; anything else - inactive
syncEnabled=0
# solr server name or IP address# [defaults to localhost if empty]
server=localhost
# solr server port# [defaults to 80 if empty]
port=8983
# URL params [mandatory]# remainder of URL
params=/select?qt=/dataimport&command=full-import&clean=false&commit=false
DataSource Parameters
All parameters are optional
endpoint - AWS Dynamo Endpoint (expert)
region - Region name
stsRoleARN - STS Role ARN to assume before connecting to Dynamo (will use credential settings). This is typically required if you need to access a different AWS account. Perhaps your dev and production aws environments are separated into 2 different accounts. The dev account might need to assume a role that is configured within production.
stsEndpoint - Custom endpoint to use for sts (expert)
stsDuration - The duration of STS alternative credentials in seconds.
accessKeyId - Explicitly set the acess key for AWS
secretKeyId - Explicitly set the secret key for AWS
credentialProfilesFile - Explicitly specify an aws profiles file to use.
credentialProfilename - Explicitly specify which profile name within the profile file to use.
credentialUseProfileDefaults - (true/false) Use the default profiles file in the default location, use NO other form of AWS authentication.
credentialUseJavaProperties - (true/false) Use java properties for authentication, use NO other form of AWS authentication.
convertType - Convert field types to the explicitly defined type in each element. Supported types are dynamo type names: N, I, S, L, BOOL
Entity Parameters
The only required entity parameter is tableName
pk - used to compare records / duplicates.
tableName - (required) the dynamo table name to retrieve records from.
keyConditionExpression - a key condition expression to use with your query (if not used the dynamo table will be scanned!)
filterExpression - a filter expression to use with your query, (applied after results are returned)
projectionExpression - a projection express to use with your query/scan (controls what fields are returned)
nameMap - (Field Prefix) When your query contains keyword values, you must use nameMap to provide alternative names, that arent' reserved. (see NameMap below)
valueMap - (Field Prefix) If you wish to inject variables into your query, you can use a Value Map to (safely) achieve this. (see ValueMap below)
Because we don't want to make any assumptions for you, the only shared value between FULL IMPORT and DELTA IMPORT is the tableName entity value.
For the DELTA IMPORT at least one Key Condition or Filter Conditon must be specified.
These entity fields are for DELTA IMPORT:
DELTAkeyConditionExpression
DELTAfilterExpression
DELTAprojectionExpression
DELTAnameMap
DELTAvalueMap
Custom Variables
Dynamo does not support actual "DATE" objects by default, for this reason people often use epoch numbers for comparison of dates if they want to search using a keyConditionExpression by creating a secondary index on a dynamo field (attribute) containing an epoch date.
We support this by helping you to build a custom keyConditionExpression using epoch seconds by provided custom variables.
For deltaimport these custom variables are provided for you:
dataimport.dynamo.last_index_time_epoch_sec
dataimport.dynamo.last_index_time_epoch_ms
The custom variables are created by reading the dataimport.last_index_time variable that Solr provides. The DataImportHandler currently assumes the date format for dataimport.last_index_time is the default of: yyyy-MM-dd HH:mm:ss - Because this is the default, you shouldn't need to change anything to make this work.
You can specify as many attributes as you want that begin with nameMap. But to make each attribute unique you must use a suffix, the suffix is arbitrary and ignored, it simply provides a unique name. Attributes like: nameMap1, nameMap2, nameMap3 are perfectly legal.
Value Maps
ValueMaps provide a way to inject values into a filterExpression or keyConditionExpression at query time.
The dynamo documentation for Solr has several examples of doing exactly this for example:
ScanSpec scanSpec =newScanSpec()
.withProjectionExpression("#yr, title, info.rating")
.withFilterExpression("#yr between :start_yr and :end_yr")
.withNameMap(newNameMap().with("#yr", "year"))
.withValueMap(newValueMap().withNumber(":start_yr", 1950).withNumber(":end_yr", 1959));
To accomplish this same query using the DataImportHandler configuration you would provide an configuration like this:
<entityprocessor="com.dhi.solr.dataimporthandler.DynamoEntityProcessor"filterExpression=""#yr between :start_yr and :end_yr" projectionExpression="#yr, title, info.rating" nameMapYear="#yr, year" valueMapStart="Int :start_yr, 1950" valueMapEnd="Int :end_yr, 1959" />
You can specify as many attributes as you want that begin with valueMap. But to make each attribute unique you must use a suffix, the suffix is arbitrary and ignored, it simply provides a unique name. Attributes like: valueMap1, valueMap2, valueMap3 are perfectly legal.
Value Map Solr Variable Injection
Because we support solr template variables in all ValueMaps, you can inject solr variables into your value that will be evaluated when the request is evaluated.
Request variables can be injected from the DataImportHandler request arguments using:
${request.[url-argument]} where url-argument is the name of the url argument passed-into the DataImport request handler.
Hopefully you're here if you want it clearly stated how to manipulate a sound-cue from C++ without relying on any blueprint. I'm going to assume just about zero knowledge, so for the more experienced you may want to skip down to the code sections.
I'm powering my computer with a bicycle..., just show me the code!: (copyable code-blocks below)
Using: USoundCue UAudioComponent from C++, C (no blueprints needed)
The final result of our pitch modulated Sound Cue
Sound cue refresher (skip me if you're familiar with a sound cue)
A sound cue represents an encapsulated group of logic around a sound or sounds. A sound cue can switch between playing multiple sounds, apply effects to the sound based on input, etc. To the engine, a sound cue is a singular object/sound, but may contain quite a lot of complexity, and mix together many different sounds. Sound cue's contain parameters just like materials contain parameters, and these parameters can be set during runtime to manipulate how the sound-cue behaves. In our case we are going to modify a modulation parameter to control the pitch of a looping sound cue.
Setup our sound cue in the editor (skip me if you have a sound cue w/parameters)
Step 1.) Create a new sound cue (content browser - Add New - sounds - sound cue)
Step 2.) Save the sound-cue with a name (in my case 'airplane-engine') and open it.
Step 3.) Add a Wave Player to the Sound Cue.
Step 4.) Add a Continuous Modulator to the Sound Cue
Step 5.) Connect Wave Player to Modulator to Output.
Step 6.) Click on the Wave Player to configure it, and set what sound it plays (.wav file)
In our case, I want the sound to loop continiously, so I check 'Looping'.
Step 7.) Configure the modulator
It's important the parameter value is set, in my case I only care about manipulating pitch at runtime, and not volume, however you can set parameters for both if you need to manipulate both during runtime. We will use this parameter name 'pitch' later in our code.
Step 8.) My real configuration for my propeller
Step 9.) Save the cue and close it...
Code Configuration
First, I'm going to place all my logic in a Pawn class that inherits from AActor... you can theoretically place your sound logic anywhere, but this example will include a pawn.
Step 1.) We need to setup some fields in our header to hold references to our objects. Ensure you use the UPROPERTY() macros, as this prevents unreal from garbage collecting the object before it should.
Create a pawn class and in your header file ensure these properties and methods are defined.
Step 2.) We need to configure our constructor to get references to and configure our sound objects.
Step 3.) We need to add our Sound Cue to our Audio Component, which you should avoid doing in the constructor, so we'll perform the same action in PostInitializeComponents()
Step 4.) When our pawn is spawned (-1 for rhyming) we need to have it start playing the Cue audio. You may want to do this based on some other event, but for my needs I want the sound to start playing on BeginPlay
Step 5.) Each frame we need to update the sound parameter pitch based on our engines speed.
