Class GcTableSyncConfiguration
java.lang.Object
edu.internet2.middleware.grouperClient.jdbc.tableSync.GcTableSyncConfiguration
an instance of this class focuses on the configuration for table sync
create an instance, set the key, and call configure
-
Nested Class Summary
Nested Classes -
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionvoidconfigureTableSync(Map<String, Object> debugMap, GcTableSync theGcTableSync, String theConfigKey, GcTableSyncSubtype theGcTableSyncSubtype) intbatch size when batching dataif doing fullSyncChangeFlag (look for a col that says if the rows are equal, e.g. a timestamp or a checksum)all columns, could be * which means all, or list of comma separated values, default to *key in config that points to this instance of table syncdatabase from keydatabase to keydatabase to or readonlydatabase to key (readonly) if large queries should be performed against a different databasegc table syncsubtype which also implies which type (full | incremental)group columnintname of a column in "FROM" table that has a sequence or last updated datetable where real time primary key and last_updated col isname of a column that has a sequence or last updated dateintnumber of bind vars in selectintminimum results expected, if this is set to -1, then it will not checkprimary key columns, * means all columns, or list of comma separated values, default to *intswitch from incremental to full if the number of groups (and records over threshold) is over this threshold i.e. needs to be over 100 groups and over 300000 recordsintswitch from incremental to full if the number of incrementals is over this thresholdswitch from incremental to full if the number of incrementals is over the threshold, this is full sync to switch to fullSyncChangeFlag, fullSyncFull, fullSyncGroupsintswitch from incremental to group (if theres a grouping col) if the number of incrementals for a certain grouptable name fromtable name fromtable name tobooleanbooleanif there are multiple source records with the same key, that is an error, this will ignore those to not be an errorbooleanif remove null characters (not null values) from strings e.g. when going to postgres (default false)retrieveConfigBoolean(String configName, boolean required) get a config name for this or dependencyretrieveConfigInt(String configName, boolean required) get a config name for this or dependencyretrieveConfigString(String configName, boolean required) get a config name for this or dependencyvoidsetBatchSize(int batchSize1) batch size when batching datavoidsetChangeFlagColumnString(String fullSyncChangeFlagColumnString1) if doing fullSyncChangeFlag (look for a col that says if the rows are equal, e.g. a timestamp or a checksum)voidsetColumnsString(String columnsString1) all columns, could be * which means all, or list of comma separated values, default to *voidsetConfigKey(String key1) key in config that points to this instance of table syncvoidsetDatabaseFrom(String databaseFrom1) database from keyvoidsetDatabaseTo(String databaseTo1) database to keyvoidsetDatabaseToReadonly(String databaseToReadonly1) database to key (readonly) if large queries should be performed against a different databasevoidsetEnabled(boolean enabled) voidsetGcTableSync(GcTableSync gcTableSync1) gc table syncvoidsetGcTableSyncSubtype(GcTableSyncSubtype gcTableSyncSubtype1) subtype which also implies which type (full | incremental)voidsetGroupColumnString(String groupColumn1) voidsetGroupingSize(int groupSize1) voidsetIgnoreDuplicateSourceKeys(boolean ignoreDuplicateSourceKeys) if there are multiple source records with the same key, that is an error, this will ignore those to not be an errorvoidsetIncrementalAllColumnsColumnString(String incrementalAllColumnsColumnString1) name of a column in "FROM" table that has a sequence or last updated datevoidsetIncrementalPrimaryKeyTable(String realTimeTable1) table where real time primary key and last_updated col isvoidsetIncrementalProgressColumnString(String incrementalAllColumnsColumnString1) name of a column that has a sequence or last updated datevoidsetMaxBindVarsInSelect(int maxBindVarsInSelect1) voidsetMinimumResultsExpected(int minimumResultsExpected1) minimum results expected, if this is set to -1, then it will not checkvoidsetPrimaryKeyColumnsString(String primaryKeyColumnsString1) primary key columns, * means all columns, or list of comma separated values, default to *voidsetRemoveNullCharactersFromStrings(boolean removeNullCharactersFromStrings) if remove null characters (not null values) from strings e.g. when going to postgres (default false)voidsetStatusDatabase(String statusDatabase1) grouperClient.syncTable.personSource.statusDatabase = awsDevvoidsetSwitchFromIncrementalToFullIfOverGroupCount(int switchFromIncrementalToFullIfOverGroupCount1) switch from incremental to full if the number of groups (and records over threshold) is over this threshold i.e. needs to be over 100 groups and over 300000 recordsvoidsetSwitchFromIncrementalToFullIfOverRecords(int switchFromIncrementalToFullIfOverRecords1) switch from incremental to full if the number of incrementals is over this thresholdvoidsetSwitchFromIncrementalToFullSubtype(GcTableSyncSubtype switchFromIncrementalToFullSubtype1) switch from incremental to full if the number of incrementals is over the threshold, this is full sync to switch to fullSyncChangeFlag, fullSyncFull, fullSyncGroupsvoidsetSwitchFromIncrementalToGroupIfOverRecordsInGroup(int switchFromIncrementalToGroupIfOverRecordsInGroup1) switch from incremental to group (if theres a grouping col) if the number of incrementals for a certain groupvoidsetTableFrom(String tableName1) table name fromvoidsetTableTo(String tableNameTo1) table name to
-
Constructor Details
-
GcTableSyncConfiguration
public GcTableSyncConfiguration()
-
-
Method Details
-
isEnabled
public boolean isEnabled() -
setEnabled
public void setEnabled(boolean enabled) -
getDatabaseFrom
database from key- Returns:
- the databaseFrom
-
getDatabaseTo
database to key- Returns:
- the databaseTo
-
getDatabaseToReadonly
database to key (readonly) if large queries should be performed against a different database- Returns:
- the databaseToReadonly
-
getDatabaseToOrReadonly
database to or readonly- Returns:
- the databaseToReadonly if exists or databaseTo
-
setDatabaseToReadonly
database to key (readonly) if large queries should be performed against a different database- Parameters:
databaseToReadonly1- the databaseToReadonly to set
-
getGcTableSyncSubtype
subtype which also implies which type (full | incremental)- Returns:
- the gcTableSyncSubtype
-
setGcTableSyncSubtype
subtype which also implies which type (full | incremental)- Parameters:
gcTableSyncSubtype1- the gcTableSyncSubtype to set
-
getMinimumResultsExpected
public int getMinimumResultsExpected()minimum results expected, if this is set to -1, then it will not check- Returns:
- the minimumResultsExpected
-
setMinimumResultsExpected
public void setMinimumResultsExpected(int minimumResultsExpected1) minimum results expected, if this is set to -1, then it will not check- Parameters:
minimumResultsExpected1- the minimumResultsExpected to set
-
getMaxBindVarsInSelect
public int getMaxBindVarsInSelect()number of bind vars in select- Returns:
- number of bind vars in select
-
setMaxBindVarsInSelect
public void setMaxBindVarsInSelect(int maxBindVarsInSelect1) - Parameters:
maxBindVarsInSelect1-
-
isIgnoreDuplicateSourceKeys
public boolean isIgnoreDuplicateSourceKeys()if there are multiple source records with the same key, that is an error, this will ignore those to not be an error- Returns:
-
setIgnoreDuplicateSourceKeys
public void setIgnoreDuplicateSourceKeys(boolean ignoreDuplicateSourceKeys) if there are multiple source records with the same key, that is an error, this will ignore those to not be an error- Parameters:
ignoreDuplicateSourceKeys-
-
isRemoveNullCharactersFromStrings
public boolean isRemoveNullCharactersFromStrings()if remove null characters (not null values) from strings e.g. when going to postgres (default false)- Returns:
-
setRemoveNullCharactersFromStrings
public void setRemoveNullCharactersFromStrings(boolean removeNullCharactersFromStrings) if remove null characters (not null values) from strings e.g. when going to postgres (default false)- Parameters:
removeNullCharactersFromStrings-
-
getBatchSize
public int getBatchSize()batch size when batching data- Returns:
- the batchSize
-
setBatchSize
public void setBatchSize(int batchSize1) batch size when batching data- Parameters:
batchSize1- the batchSize to set
-
getGroupingSize
public int getGroupingSize()- Returns:
- the groupSize
-
getChangeFlagColumnString
if doing fullSyncChangeFlag (look for a col that says if the rows are equal, e.g. a timestamp or a checksum)- Returns:
- col
-
setChangeFlagColumnString
if doing fullSyncChangeFlag (look for a col that says if the rows are equal, e.g. a timestamp or a checksum)- Parameters:
fullSyncChangeFlagColumnString1-
-
getIncrementalProgressColumnString
name of a column that has a sequence or last updated date- Returns:
- incremental col
-
setIncrementalProgressColumnString
name of a column that has a sequence or last updated date- Parameters:
incrementalAllColumnsColumnString1-
-
retrieveConfigInt
get a config name for this or dependency- Parameters:
configName-required-- Returns:
- the config
-
retrieveConfigBoolean
get a config name for this or dependency- Parameters:
configName-required-- Returns:
- the config
-
retrieveConfigString
get a config name for this or dependency- Parameters:
configName-required-- Returns:
- the config
-
getIncrementalAllColumnsColumnString
name of a column in "FROM" table that has a sequence or last updated date- Returns:
- column
-
setIncrementalAllColumnsColumnString
name of a column in "FROM" table that has a sequence or last updated date- Parameters:
incrementalAllColumnsColumnString1-
-
getPrimaryKeyColumnsString
primary key columns, * means all columns, or list of comma separated values, default to *- Returns:
- the primaryKeyColumnsString
-
setPrimaryKeyColumnsString
primary key columns, * means all columns, or list of comma separated values, default to *- Parameters:
primaryKeyColumnsString1- the primaryKeyColumnsString to set
-
getColumnsString
all columns, could be * which means all, or list of comma separated values, default to *- Returns:
- the columnsString
-
setColumnsString
all columns, could be * which means all, or list of comma separated values, default to *- Parameters:
columnsString1- the columnsString to set
-
getGcTableSync
gc table sync- Returns:
- table sync
-
setGcTableSync
gc table sync- Parameters:
gcTableSync1-
-
configureTableSync
public void configureTableSync(Map<String, Object> debugMap, GcTableSync theGcTableSync, String theConfigKey, GcTableSyncSubtype theGcTableSyncSubtype) - Parameters:
debugMap-configKey-gcTableSyncSubtype-
-
getSwitchFromIncrementalToFullIfOverGroupCount
public int getSwitchFromIncrementalToFullIfOverGroupCount()switch from incremental to full if the number of groups (and records over threshold) is over this threshold i.e. needs to be over 100 groups and over 300000 records- Returns:
- threshold
-
setSwitchFromIncrementalToFullIfOverGroupCount
public void setSwitchFromIncrementalToFullIfOverGroupCount(int switchFromIncrementalToFullIfOverGroupCount1) switch from incremental to full if the number of groups (and records over threshold) is over this threshold i.e. needs to be over 100 groups and over 300000 records- Parameters:
switchFromIncrementalToFullIfOverGroupCount1-
-
getSwitchFromIncrementalToGroupIfOverRecordsInGroup
public int getSwitchFromIncrementalToGroupIfOverRecordsInGroup()switch from incremental to group (if theres a grouping col) if the number of incrementals for a certain group- Returns:
- threshold
-
setSwitchFromIncrementalToGroupIfOverRecordsInGroup
public void setSwitchFromIncrementalToGroupIfOverRecordsInGroup(int switchFromIncrementalToGroupIfOverRecordsInGroup1) switch from incremental to group (if theres a grouping col) if the number of incrementals for a certain group- Parameters:
switchFromIncrementalToGroupIfOverRecordsInGroup1-
-
getSwitchFromIncrementalToFullSubtype
switch from incremental to full if the number of incrementals is over the threshold, this is full sync to switch to fullSyncChangeFlag, fullSyncFull, fullSyncGroups- Returns:
- type
-
setSwitchFromIncrementalToFullSubtype
public void setSwitchFromIncrementalToFullSubtype(GcTableSyncSubtype switchFromIncrementalToFullSubtype1) switch from incremental to full if the number of incrementals is over the threshold, this is full sync to switch to fullSyncChangeFlag, fullSyncFull, fullSyncGroups- Parameters:
switchFromIncrementalToFullSubtype1-
-
getSwitchFromIncrementalToFullIfOverRecords
public int getSwitchFromIncrementalToFullIfOverRecords()switch from incremental to full if the number of incrementals is over this threshold- Returns:
- threshold
-
setSwitchFromIncrementalToFullIfOverRecords
public void setSwitchFromIncrementalToFullIfOverRecords(int switchFromIncrementalToFullIfOverRecords1) switch from incremental to full if the number of incrementals is over this threshold- Parameters:
switchFromIncrementalToFullIfOverRecords1-
-
getConfigKey
key in config that points to this instance of table sync- Returns:
- the key
-
getIncrementalPrimaryKeyTable
table where real time primary key and last_updated col is- Returns:
- the realTimeTable
-
getStatusDatabase
- Returns:
- the statusDatabase
-
getTableFrom
table name from- Returns:
- the tableName
-
getTableFromType
table name from- Returns:
- the tableName
-
getTableTo
table name to- Returns:
- the tableNameTo
-
getGroupColumnString
group column- Returns:
- the groupColumn
-
setDatabaseFrom
database from key- Parameters:
databaseFrom1- the databaseFrom to set
-
setDatabaseTo
database to key- Parameters:
databaseTo1- the databaseTo to set
-
setGroupColumnString
- Parameters:
groupColumn1- the groupColumn to set
-
setGroupingSize
public void setGroupingSize(int groupSize1) - Parameters:
groupSize1- the groupSize to set
-
setConfigKey
key in config that points to this instance of table sync- Parameters:
key1- the key to set
-
setIncrementalPrimaryKeyTable
table where real time primary key and last_updated col is- Parameters:
realTimeTable1- the realTimeTable to set
-
setStatusDatabase
grouperClient.syncTable.personSource.statusDatabase = awsDev- Parameters:
statusDatabase1- the statusDatabase to set
-
setTableFrom
table name from- Parameters:
tableName1- the tableName to set
-
setTableTo
table name to- Parameters:
tableNameTo1- the tableNameTo to set
-