I wanted a quick little process of checking that a tulip connector host is up and the database is returning results. First, need to get a little selenium in my diet. You can use an already created https://github.com/alvaroseparovich/AWS-LAMBDA-LAYER-Selenium lambda layer or create your own https://dev.to/awscommunity-asean/creating-an-api-that-runs-selenium-via-aws-lambda-3ck3

Although both of those use Python 3.6 (looking around there is much angst about matching the right version of python with the right version of the chrome-driver etc..)

So make yourself a Selenium Layer.

Make yourself a Tulip Connector Supervisor user. We will use those credentials to sign in and trigger the Test. Put those credentials into SSM, create an IAM policy to grant RO access to them.

Make a heartbeat connector function and save the url.

During testing I ran a local version of selenium

https://jonathansoma.com/lede/foundations-2018/classes/selenium/selenium-windows-install/

You have to match your chromedriver to your version of chrome, but once it starts working it certainly is slick! When its being automated chrome shows this banner which is neat!

This was useful in trying to figure out how to grab the various fields of the login page.

So, import selenium and get your values from SSM

I also created a separate file (connection_list) to store a dictionary of site and connection url

connection_list.py example

tulip_connection = {'example1'          :'https://example1.tulip.co/connector/ABCD/function/EFG', 
                    'example2'          :'https://example2.tulip.co/connector/ABCD/function/EFG'
}

So create your python 3.6 lambda, set the Lambda Function’s Memory to at least 384 mb, and the time out to at least 30 sec according to alvaroseparovich

# requires selenium layer

import starter as st
import connection_list
from connection_list import tulip_connection
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException
import boto3
import os
import logging

runtime_region = os.environ['AWS_REGION']
client = boto3.client('iot-data', region_name=runtime_region)
logger = logging.getLogger()
logger.setLevel(logging.INFO)
logger.info('COLD START')

# lambda needs access to ssm  
# read only to /tulip-selenium-heartbeat/*

session = boto3.Session(region_name=runtime_region)
ssm = session.client('ssm')
ssm_username = ssm.get_parameter(Name='/tulip-selenium-heartbeat/username', WithDecryption=True)['Parameter']['Value']
ssm_password = ssm.get_parameter(Name='/tulip-selenium-heartbeat/password', WithDecryption=True)['Parameter']['Value']
logger.info("Got SSM")

So now for the event pass it

{
  "tulip_connector": "example1"
}

The body of the lambda_handler will (1) connect to the url and this will take you to the tulip sign in (2) it will wait for the data-testid=’login-username’ to appear (this tripped me up for a while since the page is dynamically generated) so you want to use the WebDriverWait (3) when found it will send the username from SSM. (4) when ata-testid=’login-password’ is located it will send the password from SSM (5) when the data-testid=’login-submit’ is located it will click it.

Next it clicks the test button, and then scrapes the values from th and td.

def lambda_handler(event, context):
    
    try:
        
        driver = st.start_drive()
    
        # use the connection_list to look up the connector function and then navigate to it
        driver.get(tulip_connection[event["tulip_connector"]])
        
        # login
        login_username = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//*[@data-testid='login-username']")))
        login_username.send_keys(ssm_username)
        login_password = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//*[@data-testid='login-password']")))
        login_password.send_keys(ssm_password)
        login_button = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//*[@data-testid='login-submit']")))
        login_button.click()
        
        # click button to test function
        
        test_button = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, '//button[normalize-space()="Test"]')))
        test_button.click()
        result_th = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, '//table/thead/tr/th[2]'))).text
        result_td = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, '//table/tbody/tr/td[2]'))).text
        result = result_th + " " + result_td
        print(result)
    
        driver.close()
    except TimeoutException as error:
        raise
    except Exception as error:
        raise
    
    return result_td

Now, assuming when you run the function and that it all works, it should bring you back

Ok, so now lets schedule this thing up.

Get ye to AWS EventBridge

Set your fixed rate and select your target lambda

And the only tricky part is to pass it the name of the connector defined in the

Assuming that is working you should start to see successful heartbeats in your cloudwatch logs.

Next up will be alert for when it fails, I’ll do that in a separate post.