Add source provider API to BaseApiAnalyzer for non-workspace analysis

CHANGES_SUMMARY.md

@@ -0,0 +1,126 @@
+# Summary of Changes for Issue #785
+
+## Problem Statement
+
+The BaseApiAnalyzer in Eclipse PDE API Tools currently requires a full Java project to perform API analysis, particularly for features like:
+- Checking @since tags in source code
+- Reporting line numbers for compatibility problems
+
+This creates challenges for use cases like:
+- Maven/Gradle mojos that work directly with JAR files
+- Standalone API analysis tools
+- Analysis scenarios where recompilation with different settings is undesirable
+
+## Solution Overview
+
+Added a new `setComponentSource()` method to `BaseApiAnalyzer` that allows external provision of source code without requiring a workspace Java project. This enables API analysis on built artifacts (JAR files) when source is available separately.
+
+## Key Changes
+
+### 1. New Public API
+
+**ISourceProvider Interface** (functional interface)
+```java
+@FunctionalInterface
+public interface ISourceProvider {
+    char[] getSource(String typeName);
+}
+```
+
+**setComponentSource Method**
+```java
+public void setComponentSource(ISourceProvider sourceProvider, Map<String, String> compilerOptions)
+```
+
+### 2. Internal Implementation Changes
+
+**New Fields:**
+- `fSourceProvider` - Stores the source provider instance
+- `fCompilerOptions` - Stores compiler options for AST parsing
+
+**New Methods:**
+- `createASTFromSource()` - Creates AST from source content without IJavaProject
+- `checkSinceTagsWithSourceProvider()` - Fallback for since tag checking without Java project
+- `checkSinceTagInAST()` - Common logic extracted for AST-based since tag checking
+
+**Modified Methods:**
+- `checkSinceTags()` - Now tries source provider when Java project is unavailable
+- `createSinceTagProblem()` - Handles null member (when using source provider)
+- `createCompatibilityProblem()` - Reports problems even without precise location info
+
+### 3. Documentation
+
+- Created `USAGE_SOURCE_PROVIDER.md` with comprehensive examples
+- Examples include: file-based sources, JAR sources, Maven mojo integration
+- Documents features, limitations, and default behavior
+
+### 4. Tests
+
+- Created `SourceProviderTest` with unit tests covering:
+  - ISourceProvider interface usage
+  - setComponentSource() method
+  - AST parsing from source content
+  - File-based source provider patterns
+
+## Design Decisions
+
+1. **Functional Interface**: ISourceProvider is a functional interface for easy lambda usage
+2. **Optional Compiler Options**: Allows callers to specify Java version or use defaults (Java 17)
+3. **Graceful Fallback**: When source provider is not set, falls back to traditional Java project approach
+4. **Backward Compatibility**: Existing code continues to work without changes
+5. **Null Handling**: All new code handles null gracefully to prevent NPEs
+6. **Limited Precision**: When using source provider, line numbers may be less precise, but problems are still reported
+
+## Benefits
+
+1. **Enables New Use Cases**: Maven/Gradle mojos can analyze JAR files with separate sources
+2. **No Recompilation**: Can analyze pre-built artifacts without rebuilding
+3. **Flexible Source Location**: Source can come from files, JARs, databases, etc.
+4. **Backward Compatible**: No breaking changes to existing API
+5. **Testable**: Can be tested without full workspace setup
+
+## Limitations
+
+When using source provider instead of Java project:
+- Line numbers may be less precise (no workspace resource information)
+- Some features that require binary type information still need compiled classes
+- Cross-references work best when all sources are available
+- Problem markers cannot be attached to workspace resources
+
+## Impact Analysis
+
+**Risk Level**: Low
+- All changes are additive (no breaking changes)
+- Existing functionality preserved with null checks
+- New code paths only activated when source provider is explicitly set
+
+**Areas Affected:**
+- BaseApiAnalyzer class (primary changes)
+- @since tag checking logic
+- Compatibility problem reporting
+
+**Testing:**
+- New unit tests added
+- Existing tests should continue to pass (backward compatible)
+- Need full test suite run to confirm no regressions
+
+## Next Steps
+
+1. **Code Review**: Review this implementation for correctness and design
+2. **Integration Testing**: Validate with actual Maven mojo implementation
+3. **Documentation Review**: Ensure documentation is clear and complete
+4. **Performance Testing**: Verify no performance regression in existing workflows
+5. **Consider Future Enhancements**: 
+   - Support for reading line numbers from debug symbols (mentioned in issue comment)
+   - More sophisticated AST caching if performance becomes an issue
+
+## Files Changed
+
+1. `apitools/org.eclipse.pde.api.tools/src/org/eclipse/pde/api/tools/internal/builder/BaseApiAnalyzer.java` - Core implementation
+2. `apitools/org.eclipse.pde.api.tools/USAGE_SOURCE_PROVIDER.md` - Usage documentation
+3. `apitools/org.eclipse.pde.api.tools.tests/src/org/eclipse/pde/api/tools/builder/tests/SourceProviderTest.java` - Unit tests
+
+## Related Issues
+
+- Issue #785: This issue
+- Issue #782: Enhanced ApiTools Mojo that motivated this feature

apitools/org.eclipse.pde.api.tools.tests/src/org/eclipse/pde/api/tools/builder/tests/SourceProviderTest.java

@@ -0,0 +1,138 @@
+/*******************************************************************************
+ * Copyright (c) 2024 Contributors to the Eclipse Foundation
+ *
+ * This program and the accompanying materials
+ * are made available under the terms of the Eclipse Public License 2.0
+ * which accompanies this distribution, and is available at
+ * https://www.eclipse.org/legal/epl-2.0/
+ *
+ * SPDX-License-Identifier: EPL-2.0
+ *
+ * Contributors:
+ *     Christoph Läubrich - initial API and implementation for issue #785
+ *******************************************************************************/
+package org.eclipse.pde.api.tools.builder.tests;
+
+import static org.junit.Assert.assertNotNull;
+import static org.junit.Assert.assertTrue;
+
+import java.util.HashMap;
+import java.util.Map;
+
+import org.eclipse.jdt.core.JavaCore;
+import org.eclipse.jdt.core.dom.AST;
+import org.eclipse.jdt.core.dom.ASTParser;
+import org.eclipse.jdt.core.dom.CompilationUnit;
+import org.eclipse.pde.api.tools.internal.builder.BaseApiAnalyzer;
+import org.junit.Test;
+
+/**
+ * Tests for the BaseApiAnalyzer source provider functionality (issue #785)
+ */
+public class SourceProviderTest {
+
+	/**
+	 * Tests that ISourceProvider can be created and used
+	 */
+	@Test
+	public void testSourceProviderInterface() {
+		// Simple source provider implementation
+		BaseApiAnalyzer.ISourceProvider provider = typeName -> {
+			if ("test.MyClass".equals(typeName)) {
+				return "package test; public class MyClass { }".toCharArray();
+			}
+			return null;
+		};
+
+		char[] source = provider.getSource("test.MyClass");
+		assertNotNull("Source should be returned", source);
+		assertTrue("Source should contain class definition", new String(source).contains("class MyClass"));
+	}
+
+	/**
+	 * Tests that setComponentSource can be called with null
+	 */
+	@Test
+	public void testSetComponentSourceNull() {
+		BaseApiAnalyzer analyzer = new BaseApiAnalyzer();
+
+		// Should not throw exception
+		analyzer.setComponentSource(null, null);
+	}
+
+	/**
+	 * Tests that setComponentSource can be called with a provider
+	 */
+	@Test
+	public void testSetComponentSourceWithProvider() {
+		BaseApiAnalyzer analyzer = new BaseApiAnalyzer();
+
+		BaseApiAnalyzer.ISourceProvider provider = typeName -> null;
+		Map<String, String> options = new HashMap<>();
+		options.put(JavaCore.COMPILER_SOURCE, JavaCore.VERSION_17);
+
+		// Should not throw exception
+		analyzer.setComponentSource(provider, options);
+	}
+
+	/**
+	 * Tests that AST can be created from source content directly
+	 */
+	@Test
+	public void testASTParsingFromSource() {
+		String sourceCode = """
+			package org.example;
+
+			/**
+			 * Test class
+			 * @since 1.0
+			 */
+			public class TestClass {
+				public void method() {
+				}
+			}
+			""";
+
+		// Create AST parser similar to how BaseApiAnalyzer does it
+		ASTParser parser = ASTParser.newParser(AST.getJLSLatest());
+		parser.setSource(sourceCode.toCharArray());
+		parser.setResolveBindings(false);
+
+		Map<String, String> options = new HashMap<>();
+		options.put(JavaCore.COMPILER_SOURCE, JavaCore.VERSION_17);
+		options.put(JavaCore.COMPILER_COMPLIANCE, JavaCore.VERSION_17);
+		options.put(JavaCore.COMPILER_DOC_COMMENT_SUPPORT, JavaCore.ENABLED);
+		parser.setCompilerOptions(options);
+		parser.setUnitName("TestClass.java");
+
+		CompilationUnit cu = (CompilationUnit) parser.createAST(null);
+		assertNotNull("Compilation unit should be created", cu);
+
+		// Verify we can navigate the AST
+		assertTrue("Should have at least one type declaration", cu.types().size() > 0);
+	}
+
+	/**
+	 * Tests a realistic source provider that simulates file-based lookup
+	 */
+	@Test
+	public void testFileBasedSourceProvider() {
+		// Simulate a source provider that would read from files
+		Map<String, String> mockSourceFiles = new HashMap<>();
+		mockSourceFiles.put("org.example.ClassA", 
+			"package org.example;\npublic class ClassA { }");
+		mockSourceFiles.put("org.example.ClassB", 
+			"package org.example;\npublic class ClassB { }");
+
+		BaseApiAnalyzer.ISourceProvider provider = typeName -> {
+			String source = mockSourceFiles.get(typeName);
+			return source != null ? source.toCharArray() : null;
+		};
+
+		// Test retrieval
+		assertNotNull("Should find ClassA", provider.getSource("org.example.ClassA"));
+		assertNotNull("Should find ClassB", provider.getSource("org.example.ClassB"));
+		assertTrue("Should return null for unknown class", 
+			provider.getSource("org.example.Unknown") == null);
+	}
+}