Using AI to Simplify Jargon for Non-Developers

It’s the end of 2024, and about half of Network Ninja is composed of developers or adjacent technical roles. That means the other half is less technical or sometimes non-technical.

LLMs, in this case Claude and GPT-4, can be used to augment these users, allowing them to leverage our technical infrastructure - and in some cases do things they could never do before.

“How is GitHub Copilot working out?”

Developers also have access to GitHub Copilot, an AI coding assistant we’ve been using for about 6 months. Unlike our chatbots, Copilot integrates directly into developers’ coding environments. It assists by completing routine code snippets and generating new functions based on our existing codebase, rather than through conversational interaction.

One of our Account Managers, Tom Reese, is not a developer. Using our internal AI assistants, he was able to extract our Copilot usage from GitHub, analyze it, and produce a bar chart illustrating code suggestions and acceptances.

Step 1

Ask two different models - Claude 3.5 Sonnet and GPT-4 - to write a command to retrieve Copilot usage data.

Screenshot of Claude 3.5 and GPT-4 providing answers to a prompt about GitHub Copilot usage. GPT-4, with its knowledge limited to early 2023, hallucinates an inaccurate answer. Claude, with its updated knowledge, provides a useful one-liner.

Step 2

Run the one-line command provided by Claude, which produces a JSON file containing Copilot usage data.

curl -H "Authorization: Bearer YOUR_ACCESS_TOKEN" \
     -H "Accept: application/vnd.github+json" \
     "https://api.github.com/orgs/YOUR_ORG_NAME/copilot/usage"

Step 3

Upload the JSON file to an AI assistant that has a sandbox to actually run code, and ask it for a summary.

User

Copilot usage JSON file. Summarize our developers’ usage of Copilot from this JSON file. Be sure to include all days in your totals, and be concise.

AI Assistant

Here is the summarized usage of Copilot across all the days provided in the JSON file:

  • Total Suggestions: 33,613
  • Total Acceptances: 8,880
  • Total Lines Suggested: 117,091
  • Total Lines Accepted: 13,386
  • Average Active Users per Day: Approximately 13.04
  • Total Chat Acceptances: 148
  • Total Chat Turns: 435
  • Average Active Chat Users per Day: Approximately 2.93
  • Total Days: 27

Step 4

Ask the AI assistant to produce a bar chart of the findings.

User

Show me a bar chart image that compares total suggestions and acceptances for particular days. Be concise.

AI Assistant

Bar chart showing GitHub Copilot usage over days. Here is bar chart that compares total suggestions and acceptances for particular days.

With a handful of prompts and a data file, our AI assistants delivered a summary and visualization of our Copilot usage. Without AI, this would’ve been a higher-effort task with developer involvement, but our assistants reduced it to a low-effort 10-minute task.

With AI Without AI
Developer needed? No Yes
Turnaround time ~10 minutes ~1 week (based on weekly dev team sprint schedule)
Level of effort required Low Low, but slow, and more $

Want to read more about AI/ML at Network Ninja?

Date

Reading Time

3 minutes

Category

Network Ninja

Are you a developer? We’re hiring! Join our team of thoughtful, talented people.