Module Syllabus
Project-Introduction
Ancient Fragments
A Virtual Reality Adventure for Learning Pictographic Scripts.
Yiming Li
Knowing to open an umbrella when it rains, and will not pick up things on the ground to eat.
TIPS Here
TIPS: Welcome to our portfolio page, you may spend quite a long journey (with a lot of content) on this page. So we recommend that you can shrink the unimportant blocks as you read to improve your reading experience.
week2 reflection
This week, we formed a group and selected a topic: Education and Entertainment.
During the seminar, we mainly did three things:
1) Get to know the group members
2) Start to think about a project topic
3) Conform the future plan
A brief introduction of our team can be referenced on the right side.
Our brand name and logo:
Below are some ideas that we discussed:
(1) Immersive Virtual Reality Environments
(2) Decoding Pictographic Scripts
(3) Gamification Elements
Our plan for the next week:
We will improve the platform design, determine the platform content, and systematically design the integration of learning content - reward features - user positive feedback mechanism.
Specification of Heuristic Evaluation (Week 3 - Week 5)
(The heuristic evaluation report can be accessed and viewed below)
Case 1 - Genshin Impact: Wondrous Shadows
- Led by Yang Gu
- Target on decryption level design, the difficulty, and immersive aspects
Case 2 - A Fisherman’s Tale
- Led by Hongbin Zhang
- Target on decryption level design, the difficulty, and immersive aspects
Case 3- Scallop
- Led by Xiang Xie
- Target on immersion in learning and interaction methods.
Case 4- Bai Ci Zhan
- Led by Haojun Shen
- Target on immersion in learning and interaction methods.
Case 5 - Monument Valley
- Led by Yiming Li
- Target on decryption level design, the difficulty, and immersive aspects
Evaluation Process
- User signs informed consent form
- Pre-test interviews
- Users conducting experiments
-
Observations and records of experimenters(experts)
- Shoot videos for post-event analysis
- Record logs
- Fill the questionnaire
- Post-test interviews
Main Observation Methods:
- Think-aloud
[What is Think-aloud?]
In a thinking-aloud test, users are asked to constantly express their thoughts while using the system - that is, simply express their thoughts in words as they move through the system interface —— Jakob Nielsen
Think loud refers to users expressing their thoughts, feelings, and actions in words when completing tasks.
[What Think-aloud can tell us?]
What part of the interface can users understand
What part of the interface cannot be understood by the user
Why users cannot understand
Did the interface we designed to operate as expected by the user
Are users surprised by what happened
Do users have any misunderstandings
- Form feedback
An example:
- Interview
The interview is unstructured and the questions of the interview are open questions
We avoid the following problems during the interview:
-
- Long questions
- Compound sentences
- Jargon and language that may not be understood
- Leading questions
- Unconscious biases
- Emotional evaluation (Gotsman et al., 2021)
Emotional assessment is based on Hanjalic's Valence-Arousal space
theory (Gotsman et al., 2021), which categorizes emotions into two dimensions: higher
valence and arousal indicate positive emotions, while lower levels signify negative emotions.
EmoFAN-VR (Gotsman et al., 2021) is a project which realizes a deep-learning model that can estimate the arousal value and valence value by evaluating a video that shows the experimenter's face during the experiments.
"This project covers emotion recognition from facial visual signals while the participant wears a virtual reality headset. We use a valence and arousal scale, a multi-dimensional emotion representation. The main contributions of this project are:
- We propose the novel EmoFAN-VR algorithm for emotion detection, trained to solve the partial face problem
- We design and record EmoVR, a novel dataset of participants displaying spontaneous emotion expressions in response to videos watched in a virtual reality environment"
Access to Heuristic Evaluation Report
-
Download CPT208.GroupB4.Report.pdf
CPT208.GroupB4.Report.pdf Details
- Sunday, 23 April 2023 [855.9KB]
Access to The Demo of Heuristic Evaluation Report
-
Download CPT208.GroupB4.ReportDemo.pdf.1
CPT208.GroupB4.ReportDemo.pdf.1 Details
- Sunday, 23 April 2023 [5MB]
week3 reflection
Task 1: Discover and define
What is the main issue to solve?
To help people understand ancient characters in depth, Oracle Bone Inscriptions for example.
What is the key area to focus upon?
Literature, linguistics, pedagogy, optics, etc.
What is the requirement(s) that has not been fulfilled by existing designs?
(1) Such science popularization products have not yet appeared or been popularized in the market. People need such a platform to satisfy their curiosity, interest and enthusiasm for learning ancient characters.
(2) Today's language learning platforms have a single mode of interaction, such as English learning platforms. A more interactive, innovative and fun way to learn text might be more effective
Academic Research:
- There are significant differences in the way pictographic and ideographic characters are remembered presented by Dhanush, hoda, and Jerry(2022).
- Another article written by Zhang and Wang(2011) proves that the human brain can remember faster with the aid of images.
- An EAP teacher confirmed the correctness of our conclusions.
Reference:
- Zhang, J., & Wang, H. (2011). The effects of orthographic depth on learning to read Chinese and English: Insights from cognitive neuroscience. Reading and Writing, 24(7), 717-732.
- Dhanush kumar Ratakonda, Hoda Mehrpouyan, Jerry Alan Fails, “Pictures are easier to remember than spellings!”: Designing and evaluating KidsPic — A graphical image-based authentication mechanism, International Journal of Child-Computer Interaction, Volume 33, 2022, 100515, ISSN 2212-8689, https://doi.org/10.1016/j.ijcci.2022.100515.
Task 2: Usability criteria
-
Our design allows users to understand and learn about Oracle in a simple and direct way, and we will simplify the operation complexity of users to the greatest extent.
-
-
-
-
-
Task 3: Atomic requirement shells
Task 4: User Stories
(1) As a user, I want to change the displayed objects in the right direction or position so that I can find out the original and structure of that ancient character.
(2) As a user, I want to interact with the software in VR, so that I can have a more realistic learning experience.
(3) As a user, I want to interact with the software on a PC or mobile phone, so that it is more convenient to access it.
(4) As a developer, I want to add a new scenario or change the original situation of an object so that the debugging or updating work might be more efficient.
(5) ...
Task 5: Persona
(1) Erick Purwanto: A middle-aged, foreign professor teaching in China, interested in ancient languages, Oracle Bone Inscriptions especially. But have no or less knowledge of Chinese culture.
(2) Yue Li: A Chinese youngster who certainly has a partial understanding of Chinese culture and an interest in learning and exploring ancient characters.
Task 6: Scenario
Erick Purwanto is a foreign professor who teaches in China. He wanted to understand the Chinese oracle bone inscriptions culture, and he used our software after a student recommended him. After Erick downloaded and opened the software, he followed the tutorial and quickly learned the core usage of the software. In addition, Erick set his own learning plan to study for one hour every Saturday morning at 10 o 'clock. The software would periodically alert Erick at this time. The software was very easy to use. Even though Erick only had an hour to learn it per week, after a few weeks of learning, he had a rich understanding of the Oracle culture and knew most of the common Oracle scripts, which made Erick highly praise the software.
Conclusion
Based on the content learned in the Lecture and the analysis of target users, we have preliminarily established our solution's Usability goals, we need to consider the product's:
- Effective to use (effectiveness) – how good it is
- Efficient to use (efficiency) – how easy / fast it is
- Safe to use (safety)
- Have good utility (utility)
- Easy to learn (learnability)
- Easy to remember how to use (memorability)
There are also User experience goals, and both desirable and undesirable aspects of the user experience need to be fully considered.
In subsequent designs, issues such as accessibility and inclusivity, as well as care for people with disabilities, need to be fully considered, although these aspects can be temporarily shelved in early design.
Studying the target group users
week4 reflection
Some real solutions in the market
- 《百词斩》
Advantages and disadvantages
- Advantages
- This is a popular software among people who are learning foreign languages. It helps users memorize words by showing different pictures related to words.
- Plans and tasks: The application provides daily tasks and learning plans to help users maintain the regularity and continuity of learning.
- Social features: The app has social features that allow users to add friends, encourage each other, and compete, increasing learning enthusiasm.
- Smart review: Based on the user's learning progress and forgetting curve, the app automatically schedules review tasks to improve learning outcomes.
- Disadvantages:
- Convenience: Baicizhan can be used on smartphones and tablets, allowing users to learn vocabulary anytime, anywhere.
- Memory through pictures is of little help to such phonographs as English and French. This method is more useful in learning Chinese and other pictographs.
Their target users:
- Student: For students who are learning foreign languages, whether they are middle school students, college students or graduate students, Hundred Words Chop can help them effectively remember and review vocabulary in their spare time.
- Foreign language lovers: For those who love learning foreign languages, "Hundred Words Chop" provides an interesting and convenient way of learning, helping them expand their vocabulary and better understand foreign language culture.
- Travelers abroad: For those who are about to travel abroad, "Hundred Words" can help them learn some basic foreign language vocabulary to meet the needs of language communication during travel.
How can they be improved:
- Provide community support: build an active user community, encourage users to share experiences, learn from each other and exchange, so as to improve learning enthusiasm and effectiveness.
- Improve user interface and interaction design: optimize the user interface and interaction design of the application to make it more intuitive and easy to use, and improve the user experience.
How can the improvements be achieved?:
Some additional interaction functions can be appended to the software design, such as interactive games or activities to motivate the users, and enhance their willingness to go on investigating the Chinese culture.
- 《Scallops 扇贝单词》
- Advantages
- This is a very popular word reciting software among students. It helps students memorize words by repeating the spelling and pronunciation of words. Some students think that it is a very efficient way to recite words and help them get good grades in English tests.
- Disadvantages
- There are still some students who think that this repetitive and boring way of memorizing words is not helpful to the improvement of their English level.
- On the contrary, this deliberate repetitive learning method makes them not interested in words.
Their target users
- Students who need to quickly improve their English vocabulary and English level.
How can they be improved
Perhaps they could try to increase the number of ways students can learn words, instead of just repeating them. For example, they can combine word memorization with games to provide users with more choices, so that they can match the habits of more people and absorb the user base.
How can the improvements be achieved?
- 《 The Chinese Sceneries 华夏风物 》
Advantages and disadvantages
- Advantages:
- This is a Chinese culture popularization app that aims to spread beautiful geographical sceneries and related comprehensive knowledge (related to climate, geography, humanity, plants etc.) to the users.
- The app 'The Chinese Sceneries' has received enormous approval and support from users who claim that it is a fantastic app that provides them with more insights about their motherland that they had never known.
- Disadvantages:
- Not being professional enough. i.e., if I want to focus on the ancient culture of someplace only it might be unclear to me because it provides a mix of multidiscipline introductions which makes it hard for me to effectively extract those that are helpful.
Their target users:
Everyone who is interested in the diversity of the Chinese culture
How can they be improved:
Maybe they can focus more on the interactions with the users because the current applications are seemingly lacking in interactions. i.e. They focus too much on single-direction culture popularization
How can the improvements be achieved?:
Some additional interaction functions can be appended to the software design, such as interactive games or activities to motivate the users, and enhance their willingness to go on investigating Chinese culture,
Key issues
- Limited Interactivity: Relying on static text, images and videos for educational content. May result in users losing interest quickly, making it difficult to maintain long-term motivation
- Insufficient Immersion: A lack of understanding of the relevant historical background may also reduce the immersive effect felt by the clients
- Narrow Scope: A narrow specification on one typical script (Egyptian hieroglyphics/Mayan glyphs) limits the users’ ability to explore multiple pictographic scripts. May restrict the platform’s appeal to a wider range of audience
My own solution
- Interactive engagement incorporations
- Include multiple pictographic scripts
-
Provide rich contextual historic & cultural information
Conclusion of the week
Based on the work of week 3 and week 4, we designed a low-fidelity prototype and carried out further user studies according to the prototype content. When designing the survey questionnaire, we considered the following aspects:
1. The reason why to use a questionnaire
- Can collect data from a large number of people, at a relatively low cost
- Can get an overview of a population of users in a short amount of time
- Surveys do not require any special equipment
- Surveys are generally approved by institutional review boards because they are typically non-intrusive
2. The deployment process
- Plan the timeline
- Design the questionnaire offline
- Program/complete online survey
- Test the survey to make sure that it behaves as you would expect
- Test it with a group that will not be part of the survey to check that the questions are clear
- Recruit participants
Prototype Iteration
We use the incremental model for prototype design, dividing the model into two modules: user service and interactive scenarios. The user service module includes the following functions:
- User registration and login
- Backend storage of user information
- Maintenance of user communities
- Maintenance of user achievement system
The interactive scene module includes the following functions:
- Scene construction (including scene design and decryption design)
- Interaction between users and scenario。
- Scene Animation Design
Prototype <Iteration 1>
Low-fidelity version [StoryBoard]
Alternatives: Low-fidelity version [Sketches of screens]
Prototype <Iteration 2>
Week5/Week6- Reflection
This week, we will deploy the presentation for the prototype, summarize and further improve all the work we have done so far, including investigation and learning (user learning, paper reading, interviews with relevant people), establish key features of the platform (light and shadow puzzle solving, virtual scenes), further enrich and organize the content of the reflection in the previous few weeks, and compile it into a phased summary of the project.
We summarize and plan our current design work based on the double diamond model proposed by the British Design Council in 2005.
1. Diamond One - Design the right thing
We have already conducted (Discover/Research), which includes information collection and divergent thinking. We have investigated user needs and identified the current shortage of related learning software in the market and the lack of user immersion. At the same time, they often have singular interactivity and overemphasize the practicality of learning. Although this is a good design, products that meet our educational and entertainment theme are relatively rare.
We conducted (Define/Thesis) on the identified issues, which involves considering and summarizing the divergent issues in the first stage, screening effective information to identify key issues. Define the problem as satisfying more user experience goals, such as Satisfying, Challenging, and Entertainment
2. Diamond II - Design things right
At (Develop/Ideation) stage, we will look for potential solutions based on the previous data foundation. To address the issue of singular interactivity, we plan to adopt interactive virtual reality scenarios to enhance user immersion. At the same time, our literature research found that using such graphical scenes to learn pictographic scripts (such as Chinese) has "inherent advantages" (go through week 3 reflection for details). At the same time, we introduce the design of light and shadow decryption, which can enable interactive VR scenes to create greater value while enhancing user immersion. In addition, we will introduce gamification, including achievement systems to implement the topic of Education and Entertainment.
In the (Deliver/Implementation) phase, we plan to iteratively develop prototype demos with higher fidelity based on the existing low-fidelity prototypes.
By implementing these novel design solutions, the platform will provide an engaging, immersive, and effective learning experience that caters to the diverse needs and preferences of its users.
Week7 - Reflection
During this week's study, we delved deeper into the concept of fidelity in prototyping. We explored the five dimensions of prototyping, which are visual, interaction, breadth, depth, and content. These dimensions are crucial in evaluating the effectiveness of a prototype.
Understanding the various dimensions of prototyping is essential in developing prototypes that accurately represent the final product. Visual fidelity involves creating a prototype that looks like the final product, while interaction fidelity focuses on creating a prototype that mimics the final product's functionality. Breadth fidelity pertains to the breadth of scenarios and use cases the prototype can handle, while depth fidelity concerns the level of detail included in the prototype. Lastly, content fidelity deals with the accuracy of the information included in the prototype.
By considering each of these dimensions, we can create prototypes that are more effective in testing and refining our ideas. It's important to note that the level of fidelity required in a prototype depends on the stage of the design process and the specific goals of the prototype. Therefore, understanding these dimensions is crucial in creating prototypes that align with our objectives.
Based on the knowledge of the lecture, we prompt a concrete design that considers more detailed issues of designing the interface. The main aspects are shown below:
- The design style should be classic and minimalistic instead of commercial.
- Design a set of widgets to enrich the user interface and menu.
- Determine the type of prototype and divide our prototype into more detailed components. (Go through <Prototype Iteration> block to access more details)
Week8/Week9 - Reflection
In this week's lecture, we have learned some additional skills on evaluating based on the fundamental evaluation skills we learned a couple of weeks ago.
Based on our existing prototypes and design concepts, we found that our work focused on Micro HCI, such as User satisfaction and Time performance, and lacked thinking about the direction of Macro HCI, such as Motivation, Empathy and Social participation. Therefore, we have decided to incorporate social systems into the product and maintain a user community. In addition, as mentioned in the reflection of Week 3, we will also consider the user experience of special groups in our future work.
We will divide the prototype evaluation for the next stage into three parts:
- Controlled settings that directly involve users: Allow users to experience the product in the development laboratory and provide feedback (the type and process of feedback can be referred to in the specification of hydrological evaluation week 3- week 5). Specifically, in this section of the product usability test, we will select 5-10 representative testers to conduct an experiment for about 30 minutes.
- Natural settings involving users: Deploy the product on users' phones or computers, allowing them to experience it in their daily lives, and provide feedback after a period of time
- Setting that does not directly involve users: consult with experts to consult and evaluate the design and prototype.
In addition, we further considered the issue of data interpretation and presentation.
- Data display: After the previous presentation, our chart color design needs to be more reasonable, and the contrast between different options needs to be high.
- Data interpretation: Consider the following aspects of the method of collecting data
- Reliability: Does the method produce the same results on separate occasions?
- Validity: Does the method measure what it is intended to measure?
- Ecological validity: Does the environment of the evaluation distort the results
- Biases: Are there biases that distort the results?
- Scope: How generalizable are the results?
Meanwhile, based on the knowledge of Experimental Design, we hope to explore and prove the conclusions obtained from the literature that have been applied to our design through our own experiments:
- Null hypothesis 1: The human brain is not more inclined to remember images than words
- Alternative hypothesis 1: The human brain is more inclined to remember images rather than words
- Null hypothesis 2: Use pictures to learn pictographic scrips, which has "congenital advantages" over ideograms
- Alternative hypothesis 2: Use pictures to learn pictographic scrips, which does not have "congenital advantages" over ideograms
We plan to use Within Subjects Design to design the experiment, as it has the advantage of "Few individuals, no individual differences" since we do not have the ability to gather too many participants. Even though this may require Counter balancing due to order issues.
Week 10/Week 11 - Iterative Reflection
In this section, we perform a thorough review of what we have done till this stage.
1. Design Goal: we have discovered the main issue we would like to solve which can be summarized as helping people understand ancient characters in depth (the Oracle Bone Inscriptions for a typical example)
Specifically, the aims can also be listed in the following bullet points:
- Wide range: design and develop a learning platform that supports learning multiple pictographic genres that are used in different countries
- Decryption: Apply light-and-shadow technology that allows users to manipulate virtual light sources to reveal hidden messages and finish the decryption.
- Immersion: Create an immersive virtual reality environment that replicates ancient sites where pictographic scripts were used for user to explore.
- Attraction: Attract at least 1,000 active users within one year and receive positive user reviews and word-of-mouth reputation.
2. Intended Target Users: The intended groups of users have been described in the following personas:
Erick Purwanto: A middle-aged, foreign professor teaching in China, interested in ancient languages, Oracle Bone Inscriptions especially. But have no or less knowledge of Chinese culture.
Yue Li: A Chinese youngster who certainly has a partial understanding of Chinese culture and an interest in learning and exploring ancient characters.
Based on the persona, we drew the Empathy map and User journey map of our solution
Empathy map
User journey map
3. Existing work and issues
The detailed content of this section can be referred to in week 4's reflection on the journal.
Here is our basic structure of iteration in this group project:
Requirements: At an early stage of this project, we have gradually made a concrete idea of the project, which is that we want to design a pictographic scripts popularization application using a special way that combines the elements of both education and entertainment.
4. Iterations
Here is a list of the iterations we have gone through the process until now:
Discovering requirements:
Previously, we have found the pain points that appear in the current market; and a further investigation of the potential clients has also revealed that they are longing for an innovative way of interaction that motivates them in the process of studying ancient characters.
Design:
Reason: Now that we have confirmed our basic ideas for this project, we need to implement an appropriate means of designing such that more users can appeal to our project.
Iteration1: We have first investigated several real applications available in the market for now, focusing on those that implement similar elements that are used in our design, for example, light-and-shadow decryption, education with a visual aid, etc., and found out to what extent they followed Schniderman's 8 golden rules for UI design.
Iteration 2: Based on the designing principles we have considered in the first iteration, we have tried our best to make our project satisfy the above requirements as much as possible. Additionally, we have also made some new processes including the implementation of several personas, and user studies to gain a deeper insight into our intended users.
iteration 3: We have further improved the interactivity of our product, by making insights into our currently intended means of interacting and making improvements and applying appropriate incorporations of different interaction methods.
Conclusion: We need to try to combine both elements of education and entertainment organically, integrate them with suitable design techniques and thus provide the users with a fully immersive and interactive experience.
Prototype:
Reason: Prototyping acts as a means to concretize our ideas into tangible models using which we could obtain feedback from the clients, and thus make continuous improvements on our product.
Iteration 1: During the period when we were figuring out what to achieve in our project, we carried out a simple demonstration of our ideas using a fairly low-fidelity medium, which is just a piece of stretch paper.
Iteration 2: We have created a gif using the software Tinkercad for a basic illustration of how our prototype works
iteration 3: In this step, we have implemented a high-fidelity version of the illustration using a more sophisticated software called Unity and simulated our expected scene to a greater extent.
Achievements: Through this iterative process we have gradually concretized our thoughts about what to show in our design and progressed gradually with the users' feedback we obtained in each iteration stage.
Evaluation:
Reason: The vital reason for evaluating is that we aim to develop a usable product that can be widely accepted by a large range of clients, thus we need to conduct an iterative and user-involved evaluation process to help us track the current process and make continuous improvements.
Iteration 1: For this step, we have utilized the relevant tools for heuristic evaluation we have seen in the lecture, with measures like distributing a questionnaire and conducting an interview with the public being taken to evaluate our design better. Then the data we collected was analyzed.
Iteration 2: Following the theoretical evaluation based on the principles we have learned in the first iteration, we have conducted a real experiment on a number of volunteers to figure out the best way to represent our product.
iteration 3: For this part, we have conducted a so-called 'field study' to investigate the users and figure out roughly their preferences and their reactions in a natural setting. Then the data has been collected and analyzed thereafter.
5. Discussion and Conclusion
Discussion: We have incrementally built up a portfolio and a prototype during the semester, with constant evaluations applied at the same time to help us develop and improve our design. However, there are also some drawbacks in terms of our techniques used, such as a lack of integration of multiple methods for implementation. In our future work, we plan to collaborate the developed products with national archaeological institutions and antiquities bureaus. The NeRF [1] technology is used to scan archaeological remains or artifacts physically into scenes and objects in virtual scenes in order to give users a more realistic experience environment.
Conclusion: Throughout this iterative process, we have realized the significance of applying a continuous manner of evaluation in terms of our development of the project, since we obtain feedback and make reflections on our work in order to improve our future performance.
[1]Mildenhall B, Srinivasan P P, Tancik M, et al. Nerf: Representing scenes as neural radiance fields for view synthesis[C]//European Conference on Computer Vision. Springer, Cham, 2020: 405-421.
Comments
Hongbin Zhang
04 May 2023, 10:46 PM
We did a really nice job!!!
Xiang Xie
04 May 2023, 10:48 PM
Nice work, handsome boys!
>3<
Haojun Shen
04 May 2023, 10:52 PM
Very good teamwork, love from China!
Acknowledgement
Thanks to all the members of Group B4, we all agree that we learned WONDERFUL knowledge in CPT208 class and did a beautiful job equipping us with the knowledge to make computers serve human society in a more human-centric way. We are proud of each other.
Thanks to CPT208 instructors Dr. Yue Li, Dr. Teng Ma. and TA Jiachen Liang for your tireless help, your guidance was indispensable.