This section presents the core findings from our two-phase user study, organized around the seven research questions (RQ1–RQ7) defined in the early stages of the project. The results include both quantitative data from Likert scales and multiple-choice questions, and qualitative insights from open-ended responses.
The aim was to measure the tool’s impact on:
- User awareness of accessibility standards.
- Real-world practices in PDF creation.
- Perceptions of usability and feature importance.
- Satisfaction and confidence post-evaluation.
Each RQ is followed by relevant data visualizations and a short analysis.
RQ1: What is the level of user awareness regarding PDF accessibility challenges?
This research question explored users’ foundational understanding of PDF accessibility, including general awareness, ability to locate relevant guidelines, and familiarity with recognized accessibility standards.
Survey Results
1: Are you aware of the aspects required to make a PDF accessible?
Pie chart showing that 51.6% of users are aware, while 48.4% are not.
2: Do you know where to find accessibility requirements for PDFs?
Pie chart showing that 69.2% of users do not know where to find requirements.
3: Do you have familiarity with any of the following accessibility guidelines/laws?
Bar chart showing recognition rates for WCAG (32.7%), ARIA (11.9%), ADA (10.7%), and that 58.5% of users did not recognize any listed guidelines.
Key Findings
- Just over half (51.6%) of participants were aware of what makes a PDF accessible.
- Nearly 7 in 10 users (69.2%) didn’t know where to locate official accessibility requirements.
- A majority (58.5%) were unfamiliar with any globally recognized accessibility standards.
Even among those with some awareness, only WCAG had notable visibility (32.7%). Most others—like ARIA, ADA, or Section 508—had recognition levels below 12%, and laws like Sugamya Bharat or the RPD Act were nearly unknown.
Insight
These results reveal a significant knowledge gap around PDF accessibility. While basic awareness is emerging, deep understanding of compliance requirements is lacking. This validated one of A11yPDF’s core goals: to not only check for issues but to educate users about the standards that define digital inclusion.
The visual reports, terminology explanations, and accessibility summaries built into the tool directly address this awareness problem—bridging a critical learning gap for both academic and everyday users.
RQ2: What is the extent to which users adopt accessibility requirements for PDF in practice?
This research question aimed to uncover whether users actively consider accessibility while creating PDFs and whether they have ever used accessibility evaluation tools prior to trying A11yPDF.
Survey Results
4:Do you consider any accessibility aspect or requirements when you create a PDF?
Likert bar chart showing that 32.7% of participants selected “1” indicating no consideration of accessibility during PDF creation.
5:Have you ever used any accessibility evaluation tools for PDF before?
88.1% of participants reported never having used any PDF accessibility tool.
Key Findings
- Nearly 1 in 3 users admitted to never thinking about accessibility when designing PDFs.
- Only 11.9% had tried any accessibility evaluation tools prior to this study.
- Among those who had, the most mentioned tools were:
- WAVE
- WCAG Validators
- Microsoft Office Accessibility Checker
- Adobe Acrobat Pro Accessibility Checker
Insight
While awareness of accessibility (RQ1) was moderate, actual adoption in practice was significantly lower. Even among users with some familiarity, few implemented any accessibility practices in their daily document workflows.
A11yPDF meets this challenge by offering an entry point that’s:
- Simple to use
- Guided visually
- Built for education as well as evaluation
This positions it not just as a tool for checking PDFs, but as a first-step solution for bridging the gap between theory and practice.
RQ3: What are the most important accessibility requirements according to users?
This research question focused on identifying which accessibility-related features users find most important when evaluating a PDF. Participants rated the importance of nine common PDF accessibility criteria on a 5-point Likert scale.t
Survey Results
6:How important are the following features in a PDF evaluation tool for you? (Very Important:1, Important:2, Moderately Important:3, Slightly Important:4, Not Important:5)
Header & Footer Clarity
- 46.5% of participants rated it as “Very Important” and 39.0% as “Important”.
- Nearly 85% of users value structural clarity for headers and footers.
Page Number Visibility
- 42.1% chose “Very Important”, 35.2% chose “Important”.
- A combined 77.3% of users consider page numbers crucial for navigability.
Page Contrast
- 37.1% rated it as “Very Important” and 45.3% as “Important”.
- Over 82% recognized contrast as an essential visual factor.
Color Blindness Accessibility
- 39.0% rated it as “Very Important”, with another 28.3% choosing “Important”.
- Shows a strong awareness of inclusive design for vision diversity.
Font Size Appropriateness
- 55.6% rated it “Very Important” — the highest rating across all features.
- Readability remains the top user priority in document accessibility.
Dyslexia-Friendly Fonts
- 29.6% said “Very Important”, 35.2% said “Important”.
- A combined 64.8% of users care about typographic accessibility for dyslexic readers.
Image Captions
- 39.0% rated as “Very Important”, and 41.5% as “Important”.
- Over 80% see value in descriptive captions for images—especially for screen readers.
Table Captions
- 42.1% rated as “Very Important”, 37.1% as “Important”.
- 79.2% want tables to be supported with semantic labeling or summaries.
URL Links
- 47.8% rated links as “Very Important”, and 34.6% as “Important”.
- Well-formed hyperlinks are considered essential for document interaction.
Key Findings
Participants most frequently rated the following features as “Very Important” (Rating 1):
- Font Size Appropriateness – 55.6%
- Dyslexia-Friendly Fonts – 47.8%
- Header & Footer Clarity – 46.5%
- Page Number Visibility – 42.1%
- Image Captions – 39.0%
- Color Blindness Accessibility – 39.0%
Even features rated lower (e.g., table captions, contrast, and URL links) still received majority support as either “Very Important” or “Important.”
Insight
These results reinforce the idea that accessibility is about more than screen reader compatibility it’s about cognitive clarity, readability, and structure. Participants emphasized features like font size, visual clarity, and support for dyslexia and color blindness.
The findings directly validate A11yPDF’s design priorities, which check for these features by default. By covering these “high-value” elements, A11yPDF aligns with both user expectations and accessibility standards, making it both relevant and practical.
RQ4: To what extent do users perceive the A11yPDF tool as helpful in evaluating PDF accessibility?
After the development and implementation of the A11yPDF tool, we wanted to evaluate its effectiveness in assessing PDF accessibility. To achieve this, participants were asked to provide feedback on two key aspects:
- The overall helpfulness of the A11yPDF tool in evaluating the accessibility of their PDFs
- The perceived accuracy of the evaluations provided by the tool
Survey Results
7:Did the A11yPDF simulation tool available on the website help you in evaluating the accessibility of your PDF?
- 50% of participants rated the tool as “Helpful”.
- 38.6% said it was “Extremely Helpful”.
- Only 10.7% were neutral
- Just 0.7% found it “Not helpful at all”.
- Over 88% of users found the tool positively impactful for accessibility evaluation.
8:In your experience, how accurate were the evaluations provided by the tool?
- 51.1% rated A11yPDF’s evaluations as “Accurate”.
- 32.4% found them “Very Accurate”.
- Small numbers said “Somewhat Accurate” (7.2%) or “Cannot evaluate” (7.2%).
- Only 2.2% felt the tool was “Not Accurate”.
- Together, 83.5% of users trusted the evaluation accuracy of the tool.
Key Findings
- A11yPDF was widely recognized as helpful and accurate, even by first-time users.
- Users not only trusted the tool’s feedback but found it actionable.
- The near-zero negative ratings ( <1% ) confirmed its general effectiveness and usability.
Insight
The overwhelmingly positive response to A11yPDF confirms that accessibility tools don’t have to be technical or intimidating. By offering intuitive design, visual results, and guided recommendations, the tool was able to engage users from all skill levels and deliver evaluations they could trust.
This strong performance across both perceived helpfulness and evaluation accuracy suggests that A11yPDF can serve as a reliable baseline accessibility checker, especially for those without prior accessibility knowledge or experience.
RQ5: Does the use of the A11yPDF tool have a measurable impact on user awareness of PDF accessibility?
With this research question, we sought to understand how exposure to the A11yPDF website influenced participants’ awareness, perspective, and confidence in designing accessible PDFs.
Participants were asked to:
- Rate the website’s educational impact
- Agree/disagree with statements about how the tool changed their design mindset
- Reflect on their preparedness to apply accessibility in practice
Survey Results
8: To what extent do you think the A11yPDF website raised your awareness and its impact on accessibility?
- 43.9% rated it 4 and 38.8% rated it 5 (Excellent).
- Over 82% found the tool significantly raised their awareness.
9:Indicate if you agree or desagree with the following statement "The A11yPDf website positively influenced my perspective on designing an accessible PDF"
- 48.8% agreed, and 38.8% totally agreed with the statement:
- The A11yPDF website positively influenced my perspective on designing an accessible PDF.
- 87.6% experienced a shift in how they view accessibility
10:To what extent has using the website influenced your approach to design and development PDF content?
- 43.2% reported a strong influence, and 32.4% noted moderate influence.
- Nearly 84% acknowledged a change in their design approach.
11:After using the website, do you feel better equipped to design or develop accessibility PDF?
- 53.2% felt “Somewhat prepared,” while 42.4% felt “Very prepared”
- A combined 95.6% now feel equipped to create more accessible content.
Key Findings
- The A11yPDF tool didn’t just evaluate — it educated.
- Users felt more informed, empowered, and ready to act.
- The interface and results presentation helped change perceptions, not just behavior.
Insight
A11yPDF proved to be more than a utility it acted as a teaching tool. By showing real-time issues in the user’s own document, it helped participants internalize accessibility principles and shift their mindset from reactive checking to proactive design.
The result? A large majority walked away not just understanding accessibility, but feeling capable of applying it exactly what inclusive tools should do.
RQ6: How does our tool compare to other tools that evaluate PDF accessibility?
This research question aimed to understand how users perceived A11yPDF compared to other accessibility evaluation tools they may have used in the past. The goal was to evaluate relative usability, usefulness, and clarity.
Survey Results
12: If you have used any accessibility PDF related accessibility tools before, how would you compare the A11yPDF tool with it?
- 51.1% of participants had never used any accessibility tool before.
- Among those who had:
- 20.1% said A11yPDF is much better
- 15.8% said it’s somewhat better
- 10.1% found it similar
- Only 2.8% rated it as somewhat or much inferior
The overwhelming majority who had previous experience rated A11yPDF more favorably than their prior tools.
Key Findings
- More than half of users were first-time accessibility tool users.
- Of those with experience, 36% preferred A11yPDF over their previous tools.
- Only 2.8% found it inferior in any way — showing strong comparative performance.
Insight
For many users, A11yPDF was their first introduction to accessibility tools — and even among experienced users, it stood out for its clarity and educational focus. Compared to technical or expensive alternatives, A11yPDF delivered a simpler, friendlier interface with actionable results.
This demonstrates the tool’s effectiveness not just in performing evaluations, but in becoming a preferred choice among users with varying levels of experience.
RQ7: Is A11yPDF a functional tool for evaluating PDF accessibility?
This research question aimed to assess the overall functionality and user experience of A11yPDF, focusing on:
- Ease of navigation
- Clarity of content
- Usefulness in understanding PDF accessibility
- General satisfaction with the tool
Participants also had opportunities to provide open-ended feedback about their experience.
Survey Results
13. Rate your overall experience with the A11yPDF website.
- 44.6% rated their experience as “Excellent”, 36.7% gave it a 4
- A combined 81.3% had a strongly positive experience using the tool.
14. Was the website easy to navigate?
- 58.3% rated it “Excellent”, 30.9% rated it 4
- AShows high usability and navigational clarity.
15. How clear and easy was it to understand the information provided on the website?
- 38.8% rated clarity as “Excellent”, 43.9% gave it a 4
- Content is accessible and understandable for most users.
16. How did you find the website’s content useful in comprehending the accessibility in PDF?
- 49.6% marked it “Excellent”, 38.8% chose 4
- Nearly 90% found the tool useful for educational purposes.
17. Overall, how satisfied are you with the PDF evaluation tool?
- 43.2% were “Very Satisfied”
- 47.5% were “Satisfied”
- 0% reported dissatisfaction
A total of 90.7% expressed high satisfaction with the tool experience.
Key Findings
- A11yPDF offers a frictionless, intuitive experience.
- Users found the content easy to follow, and the site simple to navigate.
- It scored especially well on its educational clarity, reinforcing its dual value as both a checker and teaching tool.
Insight
The strong scores across satisfaction, usability, and comprehension confirm that A11yPDF succeeds where many accessibility tools fall short — it’s not just functional, but also friendly and informative.
It proves that accessibility tools don’t have to be complicated. With the right UI and content structure, they can support both experts and beginners, guiding them toward better practices without overwhelming them.
Together, these findings demonstrate that A11yPDF succeeded in its dual mission: to evaluate accessibility accurately and to empower users to understand and improve their own documents. From raising awareness to shifting design behavior, the tool proved valuable to both novice and experienced users. These insights not only validated our design decisions but also revealed future opportunities to scale A11yPDF’s impact.
The development and deployment of A11yPDF resulted in measurable progress in improving accessibility awareness, behavior, and confidence among users. Through carefully designed surveys and evaluations, the study validated the tool’s effectiveness not only as a technical checker, but also as a practical learning platform.
1. Significant Improvement in Accessibility Awareness
Before using A11yPDF:
- Only 32.7% of participants said they considered accessibility when creating PDFs
- 88.1% had never used any accessibility evaluation tool
- Awareness of standards like WCAG, ARIA, and ADA was very limited
After using A11yPDF:
- 82% reported an increased understanding of PDF accessibility
- 95.6% felt better equipped to develop accessible content
- Participants demonstrated improved ability to recognize and prioritize key accessibility issues such as font clarity, alt text, and link readability
A11yPDF successfully addressed a widespread awareness gap by making accessibility concepts discoverable through interactive use, not passive instruction.
2. Positive Shift in Design Behavior and Confidence
A11yPDF influenced how users approached accessibility in their design and content creation process. Many respondents noted that:
- They began noticing accessibility issues they had previously ignored
- The visual feedback helped them understand why certain features were problematic
- They felt more motivated to apply accessibility principles in future projects
Additionally, survey responses confirmed that the tool influenced not only knowledge acquisition but also behavioral change.
Real-time, guided interaction proved more effective than traditional compliance checklists in driving user reflection and design awareness.
3. Strong Usability and User Satisfaction
Usability scores across all dimensions were consistently high:
- 90% of users found the site easy to navigate
- 90.7% reported overall satisfaction with the experience
- 85%+ would use A11yPDF again in the future
- The visual feedback and page-level reports were frequently highlighted as strengths
Users also appreciated the clean interface, plain-language descriptions, and section-specific reporting options.
The user interface directly contributed to increased engagement, trust in the results, and repeat usability.
4. A Learning Tool Embedded in a Functional Workflow
Participants described A11yPDF not just as a tool to check compliance but as a way to learn and self-correct. The post-survey revealed that:
- 87.6% agreed the tool changed their perspective on accessible design
- 49.6% rated it “Excellent” in helping them understand accessibility
- Many cited the tool as their first experience with applied accessibility evaluation
By embedding education directly into the tool’s flow (via tooltips, info panels, and report feedback), A11yPDF turned passive users into active learners.
5. Real-World Applicability and Broader Use Cases
A11yPDF was well-received by both academic and casual users. The platform demonstrated potential across multiple contexts, including:
- University-level coursework on HCI and inclusive design
- Content creation teams seeking quick audit tools
- Designers unfamiliar with formal accessibility standardsn
The ability to export reports for individual sections or the entire document made it particularly useful for iterative design and feedback.
A11yPDF fills an essential gap between professional-grade tools (often complex or costly) and beginner-friendly learning environments.
The study confirmed that A11yPDF is more than a diagnostic tool it is a catalyst for learning and behavioral change. It succeeded in delivering a guided, empowering experience that shifted user mindsets and improved design practices. By prioritizing usability, education, and actionable reporting, the tool helped users not just fix their documents but understand how to design with inclusion in mind.
