This article provides an overview to get started with the Github connector in Peliqan. Please contact support if you have any additional questions or remarks.

Contents

Connect

In peliqan, go to Connections, click on Add new. Find Github in the list and select it. Click on the Connect button. This will open Github and allow you to authorize access for Peliqan. Once that is done, you will return to Peliqan.

Explore & Combine

The Github connector in Peliqan is currently a “writeback”-only connector, this means it can be used in Python scripts but there are no out-of-the-box ETL pipelines that will sync data from Github into tables in the Peliqan data warehouse.

Writeback (Python scripts)

Below are examples of how to read and write from and to Github from Peliqan Python scripts (data apps).

Write file to Github (commit)

Here’s an example to commit files to Github from a script in Peliqan:

<aside> ☝ Note that in order to update an existing file, you need the SHA. Use file_get() to retrieve the SHA of the file first, and include it when updating the file.

</aside>

import base64, json

github_api = pq.connect('Github')

content = { "test_key": "test content" }
content_string = json.dumps(content)
base64_content = base64.b64encode(bytes(content_string, 'utf-8')).decode('utf-8')

file = {
    "path": "test_folder/test_file.json",
    "repo": "my_repo",
    "owner": "my_github_username",
    "message": "This is a commit from Peliqan script",
    "base64_content": base64_content,
    "committer_name": "John",
    "committer_email": "[email protected]"
}

get_file = github_api.get('file', file)

if 'sha' in get_file:
    st.text("Updating file")
    file['sha'] = get_file['sha']
    result = github_api.update('file', file)
else:
    st.text("Adding new file")
    result = github_api.add('file', file)

st.text(result["status"])

Read file from Github

import base64

github_api = pq.connect('Github')
file = {
    "owner": "ownername",
    "repo": "reponame",
    "path": "my_script.py",
}
result = github_api.get('file', file)
content = base64.b64decode(result["content"]).decode("utf-8")
st.write(content)

Peliqan CI/CD using Github

Using the Github connector, you can build CI/CD pipelines that e.g. push updates of data models (queries) and Python scripts from Peliqan to Github, and that pull in updates from Github into Peliqan.