[{"_1":2,"_134":-5,"_135":-5},"loaderData",{"_3":4,"_132":-5,"_133":-5},"routes/_app",{"_5":6,"_98":99,"_117":118},"posts",[7,37,54,69,82],{"_8":9,"_10":11,"_12":13,"_14":15,"_20":21,"_22":23,"_24":25},"slug","Electryon","title","Building Electryon: Where Monopoly Meets Web3","date","2025-01-14T00:00:00.000Z","tags",[16,17,18,19],"solana","rust","web3","hackathon","excerpt","How we built a Solana-based investment game that won first place at Taipei Blockchain Week 2024","content","

Ever wondered what would happen if you combined the classic game of Monopoly with real-world business investments? That's exactly what we did with Electryon, a gamified investment platform that recently won first place on the Solana Zeus Bounty at Taipei Blockchain Week 2024's hackathon, earning us 8888 $ZEUS (approximately £4000)!

The Vision

Electryon transforms the traditional investment experience into an exciting board game adventure. Built on the Solana blockchain, it allows players to invest in real local businesses while enjoying familiar Monopoly-style gameplay mechanics.

Key Features

🎲 Gameplay Meets Investment

Instead of just buying fictional properties, players can acquire fractional ownership in actual local businesses. Roll the dice, move around the board, and when you land on a business property, you can invest using SOL (Solana's native cryptocurrency).

🏪 Real Business Integration

Each property on the board represents a real business. When you invest, you receive business-specific tokens backed by SOL, creating a direct connection between gameplay and real-world value.

🎮 Play-to-Earn Mechanics

Players can earn tokens through various gameplay mechanics, which can then be:

Exchanged back to SOL
Used at partner businesses
Traded with other players

🖼️ NFT Integration

We've added a collecting aspect through:

Digital property cards as NFTs
Exclusive NFTs for completing achievements
Special edition collectibles for game milestones

Technical Implementation

Technology Stack

We built Electryon using:

Solana Blockchain: For its lightning-fast transactions and minimal fees
Rust: For implementing smart contracts and token logic
Remix React: For building a responsive and interactive frontend
Web3.js: For blockchain integration

Project Architecture

The project is split into two main components:

Frontend (/client)
\n
- Built with Remix React
- Integrates @solana/web3.js for blockchain interactions
- Handles game logic and user interface
- Manages wallet connections and transactions
\n
Smart Contracts (/coin)
\n
- Implemented in Rust
- Manages token contracts
- Handles liquidity pools
- Controls business token minting and burning
\n

The Hackathon Experience

During Taipei Blockchain Week 2024, our team worked intensively to bring Electryon to life. The challenge was to create something that would make investing more accessible and engaging while providing real value to local businesses.

Our efforts paid off when we were awarded first place, receiving 8888 $ZEUS tokens. The judges were particularly impressed with:

The innovative combination of gaming and investment
Real-world business integration
Technical implementation on Solana
Potential for community impact

Try It Yourself

Want to experience Electryon? You can:

What's Next?

We're currently working on:

Expanding our network of partner businesses
Adding more gameplay features
Implementing additional token utilities
Improving the user experience

Get Involved

Interested in contributing or learning more? Connect with us:

Oliver (Software Developer)

Telegram: @loreviq
Email: oliver.tj@oliver.tj

Steven (Product / Business)

Telegram: @Xoao3
Twitter: @xoaonft

Note: Electryon is currently in testing phase. While you can try it out today, please be aware that it's just a hackathon project and is no longer under development.

Want to contribute or report issues? Check out our GitHub repository and feel free to submit pull requests or open issues!

\n","links",[26,31,34],{"_27":28,"_29":30},"name","GitHub","url","https://github.com/LoreviQ/electryon",{"_27":32,"_29":33},"Live Demo","https://electryon-six.vercel.app/",{"_27":35,"_29":36},"Hackathon","https://earn.superteam.fun/listing/virtual-hackathon-bonus-bounty-tbw-2024/",{"_8":38,"_10":39,"_12":40,"_14":41,"_20":46,"_22":47,"_24":48},"SketchSlides","Building SketchSlides - A Free, Open-Source Drawing Practice Tool","2024-12-30T00:00:00.000Z",[42,43,44,45],"typescript","react","pwa","web development","I built SketchSlides as a free, open-source drawing practice tool. It's a simple, modern PWA that allows you to practice figure drawing with reference images.","

Recently, I've been learning to draw, and one of the fundamental practices in learning figure drawing is doing quick sketches from reference images. The concept is simple: look at an image for a short time - often starting with just 30 seconds - and try to capture the essence of the pose. This trains your brain to quickly identify the core elements of a figure and helps develop the muscle memory needed for confident strokes.

The Problem with Existing Tools

When I started looking for tools to help with this practice, I found myself frustrated with the available options. Most free applications were incredibly basic - just simple image viewers with a timer slapped on. The few full-featured options that existed were surprisingly expensive, often asking $15 or more for what is, essentially, a glorified slideshow player.

The paid apps did offer nice features - custom intervals, image flipping, grid overlays - but nothing that justified their price tag. Even worse, many of these applications required constant internet connectivity or had clunky, dated interfaces that felt like they were designed in the early 2000s.

Building a Modern Alternative

I decided to build SketchSlides as a modern, free alternative. My goals were simple:

Make it completely free and open source
Ensure it works offline
Create a clean, intuitive interface
Include all the features of paid alternatives

I chose to build it as a Progressive Web App (PWA) using React, TypeScript, and Vite. The PWA approach was perfect for this use case - it allows users to install the app like a native application, works offline after the first visit, and loads instantly with minimal overhead. Plus, it works on any device with a modern browser, eliminating the need for platform-specific development.

The Technical Journey

The development process was an interesting exercise in modern web development. I wanted to take advantage of the latest browser APIs while still maintaining broad compatibility. One of the most interesting challenges was implementing the file system handling.

Modern browsers now support the File System Access API, which allows web applications to interact with the user's file system in a secure way. This enabled me to implement a folder picker that lets users select an entire directory of reference images - a much better experience than selecting files individually. However, not all browsers support this yet, so I had to implement a fallback to traditional file picking.

The timing system was another interesting challenge. I wanted to support both simple fixed intervals (30 seconds, 1 minute, etc.) and complex custom schedules where users could define sequences of different intervals. This led to creating a flexible scheduling system that could handle both use cases while remaining intuitive for users.

Features Born from Experience

As I used the app for my own practice, I kept adding features that I found useful:

The image grid overlay was one of the first additions. When learning to draw, understanding proportions is crucial, and having a 3x3 grid helps immensely with composition and proportion checking. I implemented this as a toggleable overlay that doesn't interfere with the image itself.

Image flipping was another feature born from practical use. Sometimes, our brains become too accustomed to drawing poses in one direction, and simply flipping the reference image horizontally can help identify mistakes in our perception. I made this a simple toggle that can be activated mid-session.

The progress bar was carefully designed to be visible but not distracting. It smoothly updates to show the remaining time in the current interval, providing visual feedback without pulling focus from the reference image.

In particular, the minimalist style was incredibly important to me. I wanted the UI to get out of the way, so the user can focus on the reference image and drawing. As such I tried to remove all borders and other distractions, resizing the app to fit the size of the reference image.

Unfortunately, this is where making it a PWA came to bite me. The title bar cannot be hidden, which is frustrating. Additionally I wanted to add always on top functionality, but this is not supported by Progressive Web Apps.

Looking Forward

SketchSlides is now my daily driver for figure drawing practice, but there's always room for improvement. I'm considering adding features like:

Built-in reference collections would be valuable for beginners who don't have their own image libraries yet. However, this raises questions about image licensing and storage that need careful consideration.

More advanced timing patterns could be interesting - perhaps a system that gradually increases intervals throughout a session, or one that adapts based on user feedback.

Community features could allow users to share their custom schedules or curated reference collections, though this would need to be balanced against the app's current simplicity and offline-first nature.

Try It Yourself

You can try SketchSlides right now at sketchslides.oliver.tj. The entire project is open source, and I welcome contributions from the community. Whether you're an artist looking for a practice tool or a developer interested in modern web development, I'd love to hear your thoughts and suggestions.

If you find the tool useful, consider supporting the project on Ko-fi. And if you're interested in the technical details or want to contribute, check out the source code on GitHub.

Have questions or suggestions? Feel free to reach out or open an issue on GitHub. I'm always happy to discuss the project or help other developers working on similar tools.

\n",[49,52],{"_27":50,"_29":51},"Deployment","https://sketchslides.oliver.tj",{"_27":28,"_29":53},"https://github.com/LoreviQ/SketchSlides",{"_8":55,"_10":56,"_12":57,"_14":58,"_20":62,"_22":63,"_24":64},"Identify","Building IDentify - A Prize Winning Solana Wallet Network Explorer","2024-11-23T00:00:00.000Z",[59,60,61,19],"python","crypto","data analysis","IDentify is a prize winning tool that visualizes connections between Solana wallets by analyzing their on-chain interactions. The project won a $500 prize and helps address a crucial challenge in blockchain transparency.","

I recently participated in a Web3 AI London Hackathon, where I built IDentify - a tool that visualizes connections between Solana wallets by analyzing their on-chain interactions. The project won a $500 prize and helped address a crucial challenge in blockchain transparency.

The Problem

While blockchain transactions are publicly visible, identifying who owns specific wallets remains a significant challenge. This "pseudonymous" nature of crypto can be exploited for scams and illicit activities. We can see everything a wallet has done, but connecting these actions to real-world entities requires complex analysis of transaction patterns and wallet relationships.

The Solution: IDentify

I built IDentify to help map these relationships. The application:

Takes any Solana wallet address as input
Analyzes recent transactions to identify connected wallets
Creates an interactive visualization of wallet relationships
Provides detailed transaction statistics and token transfers
Helps identify patterns and potential connections between wallets

You can try it here: IDentify Live Demo
Source code: GitHub Repository

Technical Implementation

I built the entire application in just a few hours using:

Python and Streamlit for the web interface
Solana RPC API for blockchain data
NetworkX & Streamlit-AGraph for network visualization
Pandas for data processing

The application visualizes wallet connections as an interactive graph, with node sizes and edge weights representing the strength of relationships between wallets.

Hackathon Experience

The Web3 AI London Hackathon provided an excellent opportunity to dive deep into blockchain development while exploring the intersection of crypto, AI, and Web3. While some panels were more engaging than others, the event offered valuable insights into current trends and challenges in the space.

The hackathon's diverse focus on crypto/AI/Web3 topics created an environment where participants could explore innovative solutions. Winning the $500 prize validated the importance of addressing blockchain transparency challenges and demonstrated my ability to quickly deliver practical solutions to real-world problems.

Looking Forward

IDentify was built as a proof of concept, but there's significant potential for expansion. Future improvements could include machine learning for behavior pattern detection, multi-hop wallet exploration, and integration with known wallet labels.

This project showcases my ability to:

Rapidly prototype and deliver functional solutions
Work with blockchain technologies and complex data visualization
Identify and address real-world challenges in the crypto space
Successfully compete in hackathon environments

The experience reinforced my passion for building tools that make blockchain technology more accessible and transparent while highlighting the importance of responsible innovation in the Web3 space.

The code is open source under the MIT license, and I welcome contributions from the community to help expand its capabilities.

\n",[65,67],{"_27":28,"_29":66},"https://github.com/LoreviQ/IDentify",{"_27":32,"_29":68},"https://identify-solana.streamlit.app/",{"_8":70,"_10":71,"_12":72,"_14":73,"_20":77,"_22":78,"_24":79},"ModelCollapse","Exploring the Dangers of Model Collapse - A Deep Dive with Python","2024-09-24T00:00:00.000Z",[59,74,61,75,76],"ai","machine learning","nlp","In a world where AI is revolutionizing industries, a hidden threat looms. What happens when AI learns from its own mistakes? In this project, I explored the phenomenon of 'model collapse', where AI models deteriorate when trained on their own outputs, a situation that could affect industries from e-commerce to finance.","

In a world where AI is revolutionizing industries, a hidden threat looms. What happens when AI learns from its own mistakes? In this project, I explored the phenomenon of 'model collapse', where AI models deteriorate when trained on their own outputs, a situation that could affect industries from e-commerce to finance.

Model collapse refers to the gradual degradation of AI models when they are trained on data they themselves generate. Over time, this feedback loop can cause models to lose accuracy and, in extreme cases, become indistinguishable from random guessing.

Focusing on sentiment analysis using Amazon reviews, I investigated how AI models perform when progressively exposed to generated data. Reviews were categorized as either positive (4-5 stars) or negative (1-2 stars), with 3-star reviews intentionally excluded to simplify the classification task and sharpen the focus on model performance.

I developed the project in Python using several libraries:

nltk (Natural Language Toolkit) for preparing the dataset and tokenizing the text.
pandas for manipulating the data.
joblib for parallelization, ensuring the project could be completed efficiently.
scikit-learn (sklearn) for AI model processing and evaluation.
matplotlib for visualizing the results.

You can find the code and additional resources on my GitHub page.

Step 1: Training the Classification Model

I began by developing a sentiment classification AI, testing it across various dataset sizes, ranging from 100 to 400,000 reviews. As anticipated, I observed an increase in accuracy with larger datasets, although the gains diminished as the size grew. This initial phase was crucial for establishing a baseline performance level.

With the baseline established, I moved on to simulate a worst-case scenario where the AI is trained exclusively on its own outputs—a process that leads to total collapse.

Why Sentiment Analysis?
Sentiment analysis is a practical and well documented AI application, making it ideal for testing model robustness. By focusing on Amazon reviews, I had access to a large, diverse dataset, and excluding 3-star reviews allowed me to simplify the analysis into clearer extremes: positive or negative sentiment.

Baseline Performance at different Training Sample Sizes\nThis graph illustrates how increasing the dataset size leads to improved accuracy, but with diminishing returns as we scale.\n $\"Baseline$

Step 2: Simulating a Total Collapse Scenario

Next, I set up a ‘total collapse scenario’. In this phase, after training the model normally to a range of accuracies, I let it generate its own training data, increasing the dataset size to 400,000 reviews. This is similar to someone trying to teach themselves by continually rereading and revising their own notes.

The results were striking: the model's performance quickly deteriorated toward randomness (a score of 0.5), regardless of how well it initially performed before the collapse.

The performance graph shows a rapid and dramatic decline, represented by a steep downward curve, plummeting towards the baseline at 0.5 which is equivalent to random guessing.

Total Collapse at different start points\nThis graph illustrates the extreme impact of total collapse, quickly reverting my AI model to randomness regardless of the quality of the model at collapse.\n $\"Total$

Step 3: Exploring Different Collapse Rates

However a total collapse scenario isn't particularly realistic in the real world. Rather, as people use AI models to generate content online a portion of training data for new models will inevitably be AI generated. What I wanted to know was how much generated training data can an AI model tolerate before it becomes an issue.

To discover this, I simulated various different 'collapse rates', where a collapse rate is the ratio of new data that is generated by AI. So with a collapse rate of 10%, for every 10 new training samples, one would be generated by the previous iteration of the model.

Starting with an initial dataset of 50,000 reviews, where the model had already reached a substantial level of accuracy, I gradually increased the dataset size to 80,000, incorporating self generated data at collapse rates ranging from 0% to 100% with intervals of 10%.

As expected, models with higher collapse rates experienced faster and more severe declines in performance. However it also revealed that every single collapse rate caused a degradation in performance, even as little as 10%. The initial model with a 50000 sample training dataset outperformed the 80000 sample model with 10% collapse rate, even though only 3000 of those sample were AI generated and it had 77000 human training samples. Since the model had approximately 85% accuracy at this point, only 450 samples would be 'incorrect' as opposed to 29550 new 'correct' samples.

This shocked me since I didn't expect the issue to be so severe. However since the model was already mostly trained at 50000 samples and had hit diminishing returns, I rationalised it as the model can only get worse with new data. Curious, I decided to investigate the effects of AI generated training data on a yet unfinished mdoel.

Model Collapse at different Collapse Rates\nThis graph shows the varying rates of collapse on my initial model. As expected, a greater collapse rate leads to faster decline.\n $\"Model$

Step 4: Investigating Model Collapse on an 'incomplete' Model

In the next phase, I focused on lower collapse rates (0% to 30% in 3% increments) and smaller sample sizes (1,000 to 10,000 reviews). During this period, the models accuracy should improve from 79% to 83% as shown with 0% collapse rate.

As shown in the below graph, a collapse rate of 18% completely offsets the new correct data. The model with a 10000 sample size training data (1620 AI generated at 18% from 1000) performed almost equal to the model with 1000 samples 0% AI generated.

Collapse rates below 18% still impacted the score, resulting in worse models than at 0%, but still performed better than the initial model with only 1000 samples. Collapse rates over 18% were worse than the inital model, making the training of a more complex model pointless beyond this point.

Interestingly, I found that a 3% collapse rate had a negligible impact on the model’s accuracy. In fact, the model's performance nearly mirrored that of a model trained entirely on original data. This finding led me to investigate the effects of smaller amounts of AI-generated data in more detail.

Model Collapse at different Collapse Rates while training\nThis graph displays the effect of model collapse on a model that still has plenty of room for improvement.\n $\"Model$

Final Step: The Impact of small quantities of AI-Generated Data

In the final stage, I wanted to investigate the effects of small collapse rates, to see if the 3% having neglible impact from the previous step was an error from small sample size. I further refined my analysis by focusing on a narrower range of collapse rates (0% to 10% in 1% increments) and cross validating my results many times, getting an accurate average.

The result was a confirmation that any inclusion of AI-generated data, no matter how small, resulted in a decline in model performance. However at small ratios (0% - 3%) the difference is small enough to be within the natural randomness caused by AI models.

For instance, the difference in accuracy between 0% and 9% model collapse from 1000 samples to 10000 sample was a mere 0.6%. This is small, but still noticable, and something that needs to be taken into account in order to maximise AI model performance.

Model Collapse at different Collapse Rates in detail\nThis graph illustrates small collapse rates, detailing the effects even minor amounts of AI generated data can have.\n $\"Model$

Real World Implications

In an age where AI is being increasingly used to generate content—from reviews to news articles, this project reveals a hidden danger. If companies rely online content to train their models, they could unknowingly allow AI generated data into their training datasets. This is further complicated by the difficultly in knowing if content is AI generated or not, with AI models made to classify if content was AI generated being inaccurate themselves. As a result, they could see a decline in performance despite increasing dataset size.

This degradation could affect industries such as online retail, which depends on accurate sentiment analysis; finance, where predictive models are crucial for decision making; and generative AI models which are frequently used in the modern day to create articles and art.

The findings underscore the importance of curating high quality, human verified data for training AI models. As AI-generated content becomes more prevalent, the risk of polluting new training datasets grows, and awareness of these issues is essential for developing robust, trustworthy systems.

Conclusion

This project not only deepened my understanding of model collapse but also emphasized the critical balance between leveraging AI and ensuring data integrity. The rise of AI-generated content makes it more important than ever to be vigilant about the quality of data used to train our models.

I look forward to continuing my exploration in this fascinating field. My future projects will likely expand this research into domains such as text generation and recommendation systems, where model collapse could pose significant risks.

Feel free to explore the graphs and datasets I generated throughout this project, which illustrate these findings in greater detail. I welcome any feedback or opportunities for collaboration.

You can find my code and all related resources on my GitHub page.

\n",[80],{"_27":28,"_29":81},"https://github.com/LoreviQ/ModelCollapse",{"_8":83,"_10":84,"_12":85,"_14":86,"_20":91,"_22":92,"_24":93},"WebNovelPlatform","Building WebNovelClient - A Full-Stack Platform for Sharing Stories","2024-08-28T00:00:00.000Z",[87,88,89,43,90],"go","javascript","api","ci/cd","WebNovelClient is a full-stack project designed to showcase my technical skills across various domains, including backend API development, frontend design, and CI/CD. Inspired by my passion for writing, I created this platform to enable users to share and host written works.","

WebNovelClient is a full-stack project designed to showcase my technical skills across various domains, including backend API development, frontend design, and CI/CD. Inspired by my passion for writing, I created this platform to enable users to share and host written works.

Github Link

You can explore a live demo of the project here: https://webnovelclient-y5hewbdc4a-nw.a.run.app/.

The application is hosted on Google Cloud Platform (GCP) with minimal resources for demonstration purposes, so you may experience some latency. This is a limitation of the current server configuration, not the underlying code. Feel free to create an account and explore the platform’s features.

Key Technologies and Methodologies

In building WebNovelClient, I chose a combination of modern technologies and methodologies to create a robust, scalable, and efficient platform. My decisions were influenced by a desire to learn new skills, research on industry standards, and the availability of cost-effective resources. Below is an overview of the key technologies used and the rationale behind each choice:

1. Backend Development with Go

Why Go?
Go (or Golang) was chosen for the backend due to its strong performance, simplicity, and concurrency model. Known for its efficiency, Go is particularly well-suited for building high-performance web applications and APIs. Its lightweight goroutines allow for efficient management of concurrent tasks, which is ideal for handling multiple user requests simultaneously.
Benefits:
- Performance: Go's compiled nature and low-level capabilities result in fast execution times, making it an excellent choice for building APIs that need to serve data quickly.
- Simplicity: Go's syntax is straightforward, making it easier to write and maintain clean, readable code. This was important for ensuring that the codebase remains manageable as the project grows.
- Strong Typing and Error Handling: Go's strong typing system and emphasis on explicit error handling help reduce bugs and increase code reliability.
\n

2. Frontend Development with React

Why React?
React was selected for the frontend due to its component-based architecture, which promotes reusability and modularity. React's popularity and active community also mean that there are abundant resources and libraries available to extend its functionality.
Benefits:
- Component-Based Architecture: React's component-based structure allowed me to build reusable UI components, which made development more efficient and the application more maintainable.
- Virtual DOM: React's Virtual DOM optimizes updates to the user interface, ensuring a smooth and responsive user experience, even in complex applications.
- Rich Ecosystem: React's vast ecosystem of tools and libraries enabled me to implement features like routing, state management, and theming with ease.
\n

3. Database Management with SQLite and Turso

Why SQLite and Turso?
Turso was chosen as the cloud provider for its cost-effective hosting options, which were particularly attractive for a demonstration project. However, Turso exclusively supports libSQL, a fork of SQLite, which dictated my choice of database engine. While SQLite is lightweight and easy to set up, I encountered several limitations that made me wish I had used PostgreSQL instead, especially as the project grew in complexity.
Benefits:
- Cost-Effectiveness: Turso’s free tier allowed me to deploy and manage the database without incurring significant costs, which was ideal for a budget-conscious project.
- Simplicity: SQLite’s serverless nature made it straightforward to implement, especially in the early stages of development. It’s self-contained, requiring minimal configuration, which suited the initial scope of the project.
\n
Challenges:
- Limitations of SQLite: As the project progressed, I encountered challenges with SQLite’s limited concurrency support and lack of advanced features like complex querying and indexing, which PostgreSQL could have handled more efficiently.
- Provider Constraints: My choice of Turso as a cloud provider limited my database options to libSQL, which, while functional, didn’t meet all the needs of the project as it scaled.
\n

4. CI/CD with GitHub Actions

Why GitHub Actions?
GitHub Actions was chosen for Continuous Integration (CI) and Continuous Deployment (CD) due to its deep integration with GitHub repositories, ease of setup, and extensive community support.
Benefits:
- Automated Testing and Deployment: GitHub Actions enabled me to automate the testing and deployment process, ensuring that only tested and verified code is deployed to production. This reduces the risk of introducing bugs and keeps the application stable.
- Scalability: As the project grows, GitHub Actions can easily scale to accommodate more complex workflows, such as multi-environment deployments or integration with other services.
- Cost-Effectiveness: GitHub Actions provides a generous free tier, which allowed me to implement CI/CD without additional costs, keeping the project within budget.
\n

5. Test-Driven Development (TDD)

Why TDD?
I implemented Test-Driven Development to ensure that the code is reliable and that new features do not break existing functionality. By writing tests before the actual code, I was able to clearly define the expected behavior of each component and function.
Benefits:
- Code Reliability: TDD helped catch bugs early in the development process, reducing the likelihood of issues in production.
- Refactoring Confidence: With a robust suite of tests in place, I could refactor the codebase confidently, knowing that any breaking changes would be detected immediately.
- Documentation: The tests themselves serve as a form of documentation, providing clear examples of how each part of the code is expected to behave.
\n

6. Cloud Hosting with Google Cloud Platform (GCP)

Why GCP?
Google Cloud Platform was selected for hosting the application due to its flexibility, scalability, and the availability of a generous free tier, making it an ideal choice for deploying a demonstration project.
Benefits:
- Scalability: GCP offers the ability to scale resources as needed, which is crucial for future growth and handling increased traffic.
- Cost-Effectiveness: The free tier allowed me to deploy the application without incurring significant costs, making it accessible for demonstration purposes.
- Integration with CI/CD: GCP integrates well with CI/CD pipelines, enabling seamless deployments from GitHub Actions.
- Seamless Integration: By hosting the frontend, backend API, and image uploads all within GCP, the integration between these components is simplified. GCP’s services are designed to work together seamlessly, reducing the complexity of cross-service communication and ensuring faster, more reliable interactions between the client, API, and storage.
\n

Summary of Technology Choices

Each technology and tool used in WebNovelClient was chosen with a balance of learning opportunities, research into industry best practices, and budget considerations in mind. Together, they contribute to a robust, maintainable, and scalable application that serves as a testament to my development skills and adaptability.

Deployment Architecture Overview

WebNovelClient is composed of three main components: the frontend client, the backend API, and the database. Each of these components is deployed and managed in a way that ensures smooth interaction and efficient data handling.

Frontend (React App):
The frontend client is built with React and hosted on GCP. It serves as the user interface, allowing users to interact with the platform. The client makes HTTP requests to the backend API to fetch or submit data.
\n
Backend (API):
The API is developed in Go and is also hosted on GCP. It acts as the intermediary between the frontend client and the database. When the frontend makes a request, the API processes it, interacts with the database if needed, and returns the appropriate response to the client. This includes handling user authentication, managing content, and generating signed URLs for image uploads.
\n
Database (Turso with libSQL):
The database is hosted on Turso and uses libSQL, a fork of SQLite. It stores all the essential data, such as user information, written works, and metadata about the images. However, due to the limitations of SQLite, images themselves are not stored directly in the database.
\n
Image Storage (GCP Cloud Storage):
For image storage, GCP’s Cloud Storage is used. When a user uploads an image, the following process occurs:
\n
1. The frontend client sends a request to the API indicating that an image needs to be uploaded.
2. The API generates a signed URL from GCP Cloud Storage and returns it to the client.
3. The client uploads the image directly to the GCP bucket using the signed URL.
4. The client then sends the image URL back to the API, which stores this URL in the database.
\n

Visualizing the Architecture

Here’s a simplified diagram of the deployment architecture:

[Frontend Client (React)]  <-->  [Backend API (Go)]  <-->  [Database (Turso, libSQL)]\n    |                                      |\n    |                                      |\n    +--> [GCP Cloud Storage (Images)] <---+\n

Flow of Data:
- Frontend to Backend: The client sends requests (e.g., user data, content) to the API.
- Backend to Database: The API interacts with the database to retrieve or store data.
- Backend to GCP Cloud Storage: For image uploads, the API generates a signed URL and facilitates image storage.
\n

This architecture ensures that the platform is modular, scalable, and able to handle various data types efficiently. Hosting both the frontend and backend on GCP simplifies deployment and integration, while using Turso for the database and GCP Cloud Storage for images balances cost with functionality.

Scalability and Future-Proofing with Separate API and Client

Splitting the frontend client and backend API into distinct components provides significant advantages for scalability and resource management. This separation allows each component to be scaled independently based on its specific needs and load, optimizing resource usage and performance.

Independent Scaling:
By decoupling the frontend client from the backend API, you can scale each component independently. For example, if the API experiences increased traffic or processing demands, you can allocate additional resources specifically to the API without affecting the frontend client. Conversely, if the client side needs to handle more user interactions or load, it can be scaled separately.
\n
Resource Optimization:
This separation ensures that resources are utilized efficiently. You avoid the inefficiency of scaling both the client and API together when only one is under increased load. This means you can optimize costs and performance by allocating resources precisely where they are needed.
\n
Flexibility in Deployment:
Separate deployments allow for more flexibility. For instance, you can deploy updates or improvements to the API without requiring simultaneous updates to the client, and vice versa. This modular approach simplifies development cycles and reduces the risk of introducing bugs or downtime in unrelated parts of the application.
\n
Future Expansion with Kubernetes:
Although Kubernetes is not used in this demonstration project, it is a powerful tool worth considering for future iterations, especially as the project scales. Kubernetes excels in managing large-scale applications with multiple containers, offering benefits such as automated scaling, efficient resource management, seamless rollouts with automatic rollbacks, and self-healing capabilities. However, for this demonstration project, which operates with a single container, the complexity and overhead of Kubernetes would be unnecessary. Its primary advantages come into play when managing more extensive deployments, which is beyond the current scope. Nonetheless, understanding how Kubernetes could be integrated into a real-world version of the project shows a readiness to leverage advanced tools for scalable and efficient application management as the project evolves.
\n

Backend Development

The API codebase is organized within the ./api folder on GitHub. The main entry point initializes and serves the server, with each endpoint managed by dedicated handler functions. A handler function decodes the request, performs an action, and returns a response. For example, the handler function postUser is the following:

func (cfg *apiConfig) postUser(w http.ResponseWriter, r *http.Request) {\n    // REQUEST\n    request, err := decodeRequest(w, r, struct {\n        Name     string `json:"name"`\n        Email    string `json:"email"`\n        Password string `json:"password"`\n    }{})\n    if err != nil {\n        respondWithError(w, http.StatusBadRequest, "failed to decode request body")\n        return\n    }\n\n    // CREATE USER\n    hash, err := bcrypt.GenerateFromPassword([]byte(request.Password), 10)\n    if err != nil {\n        log.Printf("Error generating password hash: %s", err)\n        w.WriteHeader(http.StatusInternalServerError)\n        return\n    }\n    user, err := cfg.DB.CreateUser(r.Context(), database.CreateUserParams{\n        ID:           uuid.New().String(),\n        CreatedAt:    time.Now().UTC().Format(time.RFC3339),\n        UpdatedAt:    time.Now().UTC().Format(time.RFC3339),\n        Name:         request.Name,\n        Email:        request.Email,\n        Passwordhash: string(hash),\n    })\n    if err != nil {\n        log.Printf("Error creating user: %s", err)\n        respondWithError(w, http.StatusInternalServerError, "Couldn't create user")\n        return\n    }\n\n    // RESPONSE\n    respondWithJSON(w, http.StatusCreated, struct {\n        ID        string `json:"id"`\n        CreatedAt string `json:"created_at"`\n        UpdatedAt string `json:"updated_at"`\n        Name      string `json:"name"`\n        Email     string `json:"email"`\n    }{\n        ID:        user.ID,\n        CreatedAt: user.CreatedAt,\n        UpdatedAt: user.UpdatedAt,\n        Name:      user.Name,\n        Email:     user.Email,\n    })\n}\n

decodeRequest() and respondWithJSON() are helper functions defined elsewhere which decode the request into a Go struct and respond with a JSON body based on a Go Struct respectively. In this case, the main section of the handler function is responsible for actually creating the user based on the request. For postUser, it generates a hash from the provided password, then creates a database entry with the users data. Other handler functions follow a similar structure.

Internal Packages

The api contains two internal package, 'auth' and 'database'.

'auth' is used for authentication purposes. It has functions that:

Authenticate users by comparing passwords to saved password hash
Issue Access tokens to users so that they don't need to constantly send passwords (short duration, non-revocable)
Authenticates provided access tokens
Issue Refresh tokens to users so that they can obtain more access tokens (remember me functionality, long duration, revocable)
Authenticates provided refresh tokens and issues new access tokens
Revoke refresh tokens
Manage the generation of unique url-safe ids to be used for UID purposes

'database' is generated by SQLC. Queries are written in SQL in the ./sql/queries directory, and converted into the database package.

Testing

I write tests for handlers before writing them, testing the expected functionality of the various endpoints. Tests can be seen in the main package with *_test.go format, executed using go test. The api is hosted at https://webnovelapi-y5hewbdc4a-nw.a.run.app (Seperate address to the client), so the endpoints can be tested independently to the client using Postman or other tools.

Database Design and Management

The Database uses libsql, a fork of SQLite hosted by Turso, who I picked for their generous free plan since this is currently just a demonstration project. Migration is handled by goose, allowing me to migrate up and down between different schema. As mentioned previously, interaction via the database is handled with SQLC, where Go code is generated from SQL wirrten in the ./sql/queries directory.

Challenges Encountered with libSQL

As the project evolved and grew in complexity, several challenges with libSQL became apparent:

Migration Difficulties:

Reversible Migrations: My migration strategy is designed to be reversible to facilitate easy rollback if issues arise. However, SQLite’s limitations in handling column operations posed challenges. Specifically, SQLite does not support dropping columns directly. Consequently, to drop a column, one must create a new table, copy data, and then drop the old table—a cumbersome manual process unsuitable for automated migration scripts.

Code Readability and Variable Management:

Variable Naming Issues: SQLite, along with SQLC, lacks support for named variables in SQL queries, which affects code readability. For instance, in the query 'GetPublishedFictions', variables are denoted by placeholders ('?'). Due to SQLite’s constraints and SQLC’s current implementation, multiple placeholders are used for the same parameter, resulting in redundant entries in the query. This limitation makes the generated code less readable and more difficult to maintain.

Example of the SQL query:

-- name: GetPublishedFictions :many\nSELECT fictions.*, users.name AS author_name\nFROM fictions\nJOIN users ON users.id = fictions.authorid\nWHERE fictions.published = 1\nAND (? IS NULL OR fictions.title LIKE '%' || ? || '%')\nAND (? IS NULL OR users.name LIKE '%' || ? || '%')\nAND (? IS NULL OR fictions.title LIKE '%' || ? || '%' OR users.name LIKE '%' || ? || '%')\nLIMIT ?\nOFFSET ?;\n

In this query, placeholders are used repetitively, leading to code that looks like this when executed:

fictions, err := cfg.DB.GetPublishedFictions(r.Context(), database.GetPublishedFictionsParams{\n\tColumn1: stringParams["title"],\n\tColumn2: stringParams["title"],\n\tColumn3: stringParams["author"],\n\tColumn4: stringParams["author"],\n\tColumn5: stringParams["keyword"],\n\tColumn6: stringParams["keyword"],\n\tColumn7: stringParams["keyword"],\n\tLimit:   limit,\n\tOffset:  offset,\n})\n

The redundancy of variable names (e.g., Column1, Column2) highlights a significant drawback of the technology choices, reflecting the challenges of managing parameters efficiently in the current setup.

These limitations with libSQL and SQLite necessitated workarounds and adjustments in the project’s development. While these challenges were manageable, they underscored the constraints of using SQLite for more complex querying needs. Future iterations of the project might benefit from transitioning to a more feature-rich database solution like PostgreSQL, which would address these limitations and offer enhanced capabilities.

Frontend Development

The client codebase is organized within the ./client folder on GitHub. I created a react app, aiming to create a modern and responsive app that uses all the endpoints programmed in the API. I did not use any React Framework, since I wanted to use this opportunity to experiment with React and program these features myself instead of merely importing packages.

The app displays a range of features, such as responsive design, dark/light modes, authentication contexts, and interactions with my API to fetch/post data. I recommend poking around the site to fully see what I've included https://webnovelclient-y5hewbdc4a-nw.a.run.app/.

CI & CD

The project uses GitHub actions to constantly check my code.

Continuous Integration (CI) is automated through GitHub Actions, which run on every pull request to the main branch. The process includes running tests (go test -cover) and performing security checks (gosec). Continuous Deployment (CD) is triggered upon successful CI completion, automatically deploying the latest builds to GCP.

Summary and Future Directions

WebNovelClient stands as a comprehensive demonstration of my ability to integrate various technologies into a cohesive, scalable platform. By employing Go for the backend, React for the frontend, and leveraging cost-effective solutions like Turso for database management and GCP for hosting, this project illustrates my capacity to navigate modern development practices and deliver a functional, user-oriented application.

This project not only showcases my skills in building and deploying a full-stack application but also reflects my commitment to continuous learning and adaptation. The choices made—ranging from technology selection to deployment strategies—were driven by a combination of personal growth objectives, industry research, and practical constraints. The result is a robust platform for hosting and sharing written works, which can be explored live here.

As the project evolves, several areas offer potential for enhancement. Future iterations could benefit from advanced features and optimizations, such as incorporating PostgreSQL for more complex database operations or integrating Kubernetes for managing larger-scale deployments. These considerations are beyond the current scope but demonstrate a readiness to apply advanced tools for scalability and efficiency in real-world applications.

The project also highlights a key takeaway: the importance of modular design and the ability to scale individual components independently. This approach not only improves resource management but also provides a flexible foundation for future growth.

In conclusion, WebNovelClient represents more than just a technical achievement; it embodies a learning journey and a testament to the adaptability required in modern software development. I am excited to continue exploring new technologies and methodologies to enhance and expand upon this project in future iterations.

Feel free to explore the source code and dive deeper into the technical details of WebNovelClient. Your feedback and insights are always welcome.

\n",[94,96],{"_27":28,"_29":95},"https://github.com/LoreviQ/WebNovelPlatform",{"_27":32,"_29":97},"https://webnovelclient-y5hewbdc4a-nw.a.run.app/","projects",[100,106,110,114],{"_27":9,"_101":102,"_29":33,"_103":9,"_8":9,"_104":21,"_14":105,"_12":13},"logo","/images/projects/electryon.webp","blogPost","description",[16,17,18,19],{"_27":38,"_101":107,"_29":108,"_103":38,"_8":38,"_104":46,"_14":109,"_12":40},"/images/projects/sketchslides.webp","https://sketchslides.oliver.tj/",[42,43,44,45],{"_27":111,"_101":112,"_29":68,"_103":55,"_8":111,"_104":62,"_14":113,"_12":57},"IDentify","/images/projects/identify.webp",[59,60,61,19],{"_27":83,"_101":115,"_29":97,"_103":83,"_8":83,"_104":91,"_14":116,"_12":85},"/images/projects/webnovel.webp",[87,88,89,43,90],"config",{"_119":120,"_121":122,"_123":124,"_125":126,"_127":128,"_129":120,"_130":131},"NAME","Oliver Jay","DOMAIN","oliver.tj","EMAIL","oliver@oliver.tj","GITHUB","LoreviQ","LINKEDIN","oliver-tj-","SITE_TITLE","SITE_DESCRIPTION","Personal website and portfolio","root","routes/_app.contact","actionData","errors"]

Let's Connect

Email

GitHub

LinkedIn