Generative Creativity Tool

Research project for the course DSGN 118 in the realm of creative productivity through AI where our team synthesized existing research and prototyped a generative creativity tool that tackled generative UX/UI style transfer.
Visit Website Prototype

Role:
UX Designer, UX Researcher
Type:
DSGN 118 Class Project
Timeline:
April - June 2023
Team:
Sabrina Mao,
Kerri Chow,
Linzy Mueller,
Jimmy Wu

Overview

Improving Diverse Inspiration Workflow for UI with AI 
AUI is a prototype that we developed in class with the "goal is to design and prototype an interactive system to address problems that impede creativity and productivity". AUI is a prototype that addresses the space in which Artificial Intelligence is able to support creativity in the realm of UX design. It aims to leverage current models of inspiration applications and generative design models to present a potential application that would streamline designer's creative process.

Problems

Current platforms for design inspiration have trouble generating diverse yet focused sources of inspiration in bulk, as they are either:

1.
Too general / directionless: Design drift causes inspiration to divert from the original focus.
2.
Too Narrow: Design fixation hinders creativity.
3.
Design Fatigue can reduce motivation.

Preliminary Research

With our preliminary research, we looked into several different research papers:
StyLit: illumination-guided example-based stylization of 3D renderings (ACM Transactions on Graphics, 2016)
Design Guidelines for Prompt Engineering Text-to-Image Generative Models (CHI '22)
GANSpiration: Balancing Targeted and Serendipitous Inspiration in User Interface Design with Style-Based Generative Adversarial Network (CHI '22)
01. Synthesizing existing research
We began doing an analysis of existing methodologies in relevant AI for HCI research papers: Stylit creates replications of your style, GANSpiration produces similar but diverse versions of your style. Novel methods: focus on light/shadows (stylit) or multi-level conditions for generating multiple UI layouts (ganspiration). Guidelines for these frameworks that allow us to use AI tools with ease, bridging the gap between human mind and machine mind.

How can we curate UI design inspiration in a way that is more beneficial and effective for our creative process?
Provide targeted sources of inspiration with AI generated images, while still preserving designer’s own creativity and avoiding hindrances such as design drift or fixation.
Analysis of AI problem-sphere
AI has the potential to be used as an effective tool for a wide variety of applications, however at this point in time, the nature of AI is that it is not always functional and prone to error. We want to hone in on a specific use case of machine learning, related to style transfer and images generation. AI models centered around style-transfer attempt to streamline and aid an artist’s creative process but current methods are limited in their ability to generate output that will always align with users’ objective. 
‍
Importance of this domain/Motivation:
AI has the potential to be used as an effective tool for a wide variety of applications, however at this point in time, the nature of AI is that it is not always functional and prone to error. We want to hone in on a specific use case of machine learning, related to style transfer and images generation. AI models centered around style-transfer attempt to streamline and aid an artist’s creative process but current methods are limited in their ability to generate output that will always align with users’ objective. 
‍
How do we address this?
Artists of different modalities need digital tools to help speed up their creative process, whether it be for content generation, aiding inspiration, or simply streamlining tedious tasks. AI can be leveraged to help these processes but we must focus on specific factors such as specificity or context to avoid impeding on workflows. 
Competitive Analysis of design inspiration

Where we want to improve on what competitors lack:

  • Interactivity and direct application
  • Targeted User Interfaces  / discoverability
  • Targeted to a designer’s specific needs and preferences
  • Intentional inspiration generation and tag along in the process

What we want to maintain from the competitors’ strengths:

  • Extensive variety of design inspiration
  • Navigation freedom / spontaneity
Annotating the existing website for strengths and possible issues (mobile site).
Web audit of existing site based on Nielsen Norman Group’s 10 usability heuristics.
Annotating the existing website for strengths and possible issues (desktop site).

Problem question:

💡  how can we curate design inspiration in a way that is more beneficial and effective in our creative process?

Ideation: Searching for a Solution

Intial concept designs and ideas:

  • More targeted outputs through AI prompt search,  guidelines, asset filtering, and component-specific generation.
  • Preventing design drift through having side by side comparisons and a smaller set of outputs.
  • Preventing design fixation through having control of output randomness (e.g. random to similar scale).
  • Personalization through focusing on designer’s intentions; having an access to generation history, and being able to save designs in curated collections.

Wireframing & lo-fi design features

In our wireframing process we decided to focus our concept on a prompt search model with the addition of a file upload option. We wanted to have this not only generate inspiration but be a playground for designers to explore and still tap into the creative process via style guide editing.

Notable features include:

  • Prompt search and file upload inputs.
  • Carousel navigation, reducing content on the page by prioritizing one screen at a time.
  • Custom design style guide with editing access.
  • History and collection pages for users to save designs.

So where is the AI part?

After gathering feedback and doing more research to into existing AI spaces we realized our current designs still struggled to bridge user goals with more streamlined AI capabilities and the current model of creative exploration could leverage more AI exploration.

We address UI inspiration by tailoring the algorithm to the user’s goal, purpose and what field of designs they want the generation to be by having onboarding questions for the user. After generating designs, we ask the user feedback if the outputs were helpful or not. In the style guide, we added generated tags which are toggleable so the output style can change for the user. For personalization purposes, we added customization of the generated outputs, including generation history and having a favorite collection.

Anticipating AI problems

  • "Results might not give user's what they need" so we added on-boarding questions for more targeted results
  • "User may not like the generated UI style" so we added customization for the style allowing more user freedom in the inspiration, and there is a helpful or not helpful feedback cta.
  • "AI has problematic work flows from misinterpreting the user’s prompt" to help with that we want to add “suggestions” for text inputs.

Design: The Road to High Fidelity

Hi-fi screens

Style-guide and Style Tag Tool

While users are able to manually alter styles and fonts to their liking, they are also able to leverage AI tools to stylize the current design into another style to explore different options and save them to history/collections.

Visual style guide

How We Met Our Goals

1.
With our more intuitive information architecture and navigation flow, approachable ‘mad libs’ fill-in section, and clearer content, users can now access important info and resources more easily.
2.
Content has become less overwhelming in style and amount, and call-to-action elements like buttons are much clearer; these invite the user to explore the pages more.
3.
By creating and adhering to a design system that represents the brand of the organization, the website now features a more coherent visual identity that creates a unique impression on viewers.

What we learned

Communication is Key
‍
Whether it be with stakeholders or other group members, communication and trust with every step of the way led to efficient and effective progress.
Design is a Bumpy Road
‍
The process of creating beautiful experiences is anything but beautiful. Embrace the chaos and try jumping at every opportunity to improve.
Deadlines are Crucial
‍
Setting dates for when we wanted to have deliverables finished held group members accountable and ensured we progressed (even if it was all mostly done the night before).
Get Feedback
‍
Our end product was greatly improved thanks to feedback from our mentors (Tarrea, Alex, & Chy), our TFSS point of contact (Aspen), and other builders.

Next Steps

In the future, possible ways we can envision expanding on this project include:

  1. Conducting more research and gathering more user feedback.
  2. Polishing this idea and iterating on more current models of AI capabilities and UI design.
  3. Potentially launching and implementing this concept into a product in collaboration with a larger team.

A Big Thank You To…

Thank you for reading this case study! Check out my other work!

Mobile Tree Map
Mobile Design
Sabramayo@gmail.com
Linkedin